How to Efficiently Read the Last 10 Lines of a 10GB Text File in C#?
Dec 29, 2024 am 02:03 AM
Retrieving the Last 10 Lines of Massive Text Files (Over 10GB)
In the realm of text processing, a common challenge is extracting the last few lines of extremely large text files. When dealing with files exceeding 10GB, traditional approaches may fall short. This article presents an effective solution to this problem using C# and offers a code snippet to demonstrate its implementation.
To efficiently retrieve the last 10 lines, the strategy involves traversing the file backwards from the end. As the number of lines may be variable, we iteratively seek backwards until 10 line breaks are encountered. Once this point is reached, we read the remaining content forward to capture the last 10 lines.
Consider the following implementation:
public static string ReadEndTokens(string path, Int64 numberOfTokens, Encoding encoding, string tokenSeparator) {
int sizeOfChar = encoding.GetByteCount("\n");
byte[] buffer = encoding.GetBytes(tokenSeparator);
using (FileStream fs = new FileStream(path, FileMode.Open)) {
Int64 tokenCount = 0;
Int64 endPosition = fs.Length / sizeOfChar;
for (Int64 position = sizeOfChar; position < endPosition; position += sizeOfChar) {
fs.Seek(-position, SeekOrigin.End);
fs.Read(buffer, 0, buffer.Length);
if (encoding.GetString(buffer) == tokenSeparator) {
tokenCount++;
if (tokenCount == numberOfTokens) {
byte[] returnBuffer = new byte[fs.Length - fs.Position];
fs.Read(returnBuffer, 0, returnBuffer.Length);
return encoding.GetString(returnBuffer);
}
}
}
// handle case where number of tokens in file is less than numberOfTokens
fs.Seek(0, SeekOrigin.Begin);
buffer = new byte[fs.Length];
fs.Read(buffer, 0, buffer.Length);
return encoding.GetString(buffer);
}
}
This code handles cases where the number of lines in the file is less than 10 and appropriately adjusts the read operation. The encoding parameter allows for customization based on the file's encoding, and tokenSeparator can be used to retrieve the last consecutive elements of a different separator.
By employing this approach, you can effectively retrieve the last 10 lines of massive text files, ensuring efficient processing and accurate results.
The above is the detailed content of How to Efficiently Read the Last 10 Lines of a 10GB Text File in C#?. For more information, please follow other related articles on the PHP Chinese website!
Hot AI Tools
Undress AI Tool
Undress images for free
Undresser.AI Undress
AI-powered app for creating realistic nude photos
AI Clothes Remover
Online AI tool for removing clothes from photos.
Clothoff.io
AI clothes remover
Video Face Swap
Swap faces in any video effortlessly with our completely free AI face swap tool!
Hot Article
Hot Tools
Notepad++7.3.1
Easy-to-use and free code editor
SublimeText3 Chinese version
Chinese version, very easy to use
Zend Studio 13.0.1
Powerful PHP integrated development environment
Dreamweaver CS6
Visual web development tools
SublimeText3 Mac version
God-level code editing software (SublimeText3)
Hot Topics
C Polymorphism : is function overloading a kind of polymorphism?
Jun 20, 2025 am 12:05 AM
Yes, function overloading is a polymorphic form in C, specifically compile-time polymorphism. 1. Function overload allows multiple functions with the same name but different parameter lists. 2. The compiler decides which function to call at compile time based on the provided parameters. 3. Unlike runtime polymorphism, function overloading has no extra overhead at runtime, and is simple to implement but less flexible.
What Are the Different Kinds of Polymorphism in C ? Explained
Jun 20, 2025 am 12:08 AM
C has two main polymorphic types: compile-time polymorphism and run-time polymorphism. 1. Compilation-time polymorphism is implemented through function overloading and templates, providing high efficiency but may lead to code bloating. 2. Runtime polymorphism is implemented through virtual functions and inheritance, providing flexibility but performance overhead.
C : Is Polymorphism really useful?
Jun 20, 2025 am 12:01 AM
Yes, polymorphisms in C are very useful. 1) It provides flexibility to allow easy addition of new types; 2) promotes code reuse and reduces duplication; 3) simplifies maintenance, making the code easier to expand and adapt to changes. Despite performance and memory management challenges, its advantages are particularly significant in complex systems.
C Destructors: Common Errors
Jun 20, 2025 am 12:12 AM
C destructorscanleadtoseveralcommonerrors.Toavoidthem:1)Preventdoubledeletionbysettingpointerstonullptrorusingsmartpointers.2)Handleexceptionsindestructorsbycatchingandloggingthem.3)Usevirtualdestructorsinbaseclassesforproperpolymorphicdestruction.4
Polymorphism in C : A Comprehensive Guide with Examples
Jun 21, 2025 am 12:11 AM
Polymorphisms in C are divided into runtime polymorphisms and compile-time polymorphisms. 1. Runtime polymorphism is implemented through virtual functions, allowing the correct method to be called dynamically at runtime. 2. Compilation-time polymorphism is implemented through function overloading and templates, providing higher performance and flexibility.
C tutorial for people who know Python
Jul 01, 2025 am 01:11 AM
People who study Python transfer to C The most direct confusion is: Why can't you write like Python? Because C, although the syntax is more complex, provides underlying control capabilities and performance advantages. 1. In terms of syntax structure, C uses curly braces {} instead of indentation to organize code blocks, and variable types must be explicitly declared; 2. In terms of type system and memory management, C does not have an automatic garbage collection mechanism, and needs to manually manage memory and pay attention to releasing resources. RAII technology can assist resource management; 3. In functions and class definitions, C needs to explicitly access modifiers, constructors and destructors, and supports advanced functions such as operator overloading; 4. In terms of standard libraries, STL provides powerful containers and algorithms, but needs to adapt to generic programming ideas; 5
What Are the Various Forms of Polymorphism in C ?
Jun 20, 2025 am 12:21 AM
C polymorphismincludescompile-time,runtime,andtemplatepolymorphism.1)Compile-timepolymorphismusesfunctionandoperatoroverloadingforefficiency.2)Runtimepolymorphismemploysvirtualfunctionsforflexibility.3)Templatepolymorphismenablesgenericprogrammingfo
C Polymorphism: Coding Style
Jun 19, 2025 am 12:25 AM
C polymorphismisuniqueduetoitscombinationofcompile-timeandruntimepolymorphism,allowingforbothefficiencyandflexibility.Toharnessitspowerstylishly:1)Usesmartpointerslikestd::unique_ptrformemorymanagement,2)Ensurebaseclasseshavevirtualdestructors,3)Emp
