Rectangle 27 0

Objective C: Reading a file line by line?


// read everything from text
NSString* fileContents = 
      [NSString stringWithContentsOfFile:fileRoot 
       encoding:NSUTF8StringEncoding error:nil];

// first, separate by new line
NSArray* allLinedStrings = 
      [fileContents componentsSeparatedByCharactersInSet:
      [NSCharacterSet newlineCharacterSet]];

// then break down even further 
NSString* strsInOneLine = 
      [allLinedStrings objectAtIndex:0];

// choose whatever input identity you have decided. in this case ;
NSArray* singleStrs = 
      [currentPointString componentsSeparatedByCharactersInSet:
      [NSCharacterSet characterSetWithCharactersInString:@";"]];
NSString* filePath = @""//file path...
NSString* fileRoot = [[NSBundle mainBundle] 
               pathForResource:filePath ofType:@"txt"];

There you have it.

This is no response to the question. The question was to read a file line by line to reduce memory usage

This will work for general reading a String from Text. If you would like to read longer text(large size of text), then use the method that other people here were mentioned such as buffered(reserve the size of the text in memory space).

i have a 70 mb file , using this code to read file doesn't hep me it increases memory linearly. can any one help me?

Note
Rectangle 27 0

Objective C: Reading a file line by line?


Since you mention CRLF (Windows) line endings: That's actually a case that breaks the Objective-C way of doing things. If you use one of the -stringWithContentsOf* methods followed by -componentsSeparatedByCharactersInSet:[NSCharacterSet newlineCharacterSet], it sees the \r and \n separately and adds a blank line after each line.

That said, the fgets solution fails on CR-only files. But those are (theoretically) rare nowadays, and fgets does work for both LF and CRLF.

The appropriate way to read text files in Cocoa/Objective-C is documented in Apple's String programming guide. The section for reading and writing files should be just what you're after. PS: What's a "line"? Two sections of a string separated by "\n"? Or "\r"? Or "\r\n"? Or maybe you're actually after paragraphs? The previously mentioned guide also includes a section on splitting a string into lines or paragraphs. (This section is called "Paragraphs and Line Breaks", and is linked to in the left-hand-side menu of the page I pointed to above. Unfortunately this site doesn't allow me to post more than one URL as I'm not a trustworthy user yet.)

To paraphrase Knuth: premature optimisation is the root of all evil. Don't simply assume that "reading the whole file into memory" is slow. Have you benchmarked it? Do you know that it actually reads the whole file into memory? Maybe it simply returns a proxy object and keeps reading behind the scenes as you consume the string? (Disclaimer: I have no idea if NSString actually does this. It conceivably could.) The point is: first go with the documented way of doing things. Then, if benchmarks show that this doesn't have the performance you desire, optimise.

Note
Rectangle 27 0

Objective C: Reading a file line by line?


// read everything from text
NSString* fileContents = 
      [NSString stringWithContentsOfFile:fileRoot 
       encoding:NSUTF8StringEncoding error:nil];

// first, separate by new line
NSArray* allLinedStrings = 
      [fileContents componentsSeparatedByCharactersInSet:
      [NSCharacterSet newlineCharacterSet]];

// then break down even further 
NSString* strsInOneLine = 
      [allLinedStrings objectAtIndex:0];

// choose whatever input identity you have decided. in this case ;
NSArray* singleStrs = 
      [currentPointString componentsSeparatedByCharactersInSet:
      [NSCharacterSet characterSetWithCharactersInString:@";"]];
NSString* filePath = @""//file path...
NSString* fileRoot = [[NSBundle mainBundle] 
               pathForResource:filePath ofType:@"txt"];

There you have it.

This is no response to the question. The question was to read a file line by line to reduce memory usage

This will work for general reading a String from Text. If you would like to read longer text (large size of text), then use the method that other people here were mentioned such as buffered (reserve the size of the text in memory space).

i have a 70 mb file , using this code to read file doesn't hep me it increases memory linearly. can any one help me?

Note
Rectangle 27 0

Objective C: Reading a file line by line?


#include <stdio.h>

NSString *readLineAsNSString(FILE *file)
{
    char buffer[4096];

    // tune this capacity to your liking -- larger buffer sizes will be faster, but
    // use more memory
    NSMutableString *result = [NSMutableString stringWithCapacity:256];

    // Read up to 4095 non-newline characters, then read and discard the newline
    int charsRead;
    do
    {
        if(fscanf(file, "%4095[^\n]%n%*c", buffer, &charsRead) == 1)
            [result appendFormat:@"%s", buffer];
        else
            break;
    } while(charsRead == 4095);

    return result;
}
FILE *file = fopen("myfile", "r");
// check for NULL
while(!feof(file))
{
    NSString *line = readLineAsNSString(file);
    // do stuff with line; line is autoreleased, so you should NOT release it (unless you also retain it beforehand)
}
fclose(file);

I added a check to get past the empty lines: int fscanResult = fscanf(file, "%4095[^\n]%n%*c", buffer, &charsRead); if(fscanResult == 1) { [result appendFormat:@"%s", buffer]; } else { if (feof(file)) { break; } else if(ferror(file) != 0) { break; } fscanf(file, "\n", nil, &charsRead); break; }

Note: I have not tested this code. Please test it before using it.

This code reads non-newline characters from the file, up to 4095 at a time. If you have a line that is longer than 4095 characters, it keeps reading until it hits a newline or end-of-file.

This is stopping early for me after 812 lines. The 812th line is "... 3 more", and that's making the reader output empty strings.

Use as follows:

how would you modify the format to accept empty lines, or rather lines consisting of a single newline character?

Note
Rectangle 27 0

Objective C: Reading a file line by line?


Since you mention CRLF (Windows) line endings: That's actually a case that breaks the Objective-C way of doing things. If you use one of the -stringWithContentsOf* methods followed by -componentsSeparatedByCharactersInSet:[NSCharacterSet newlineCharacterSet], it sees the \r and \n separately and adds a blank line after each line.

That said, the fgets solution fails on CR-only files. But those are (theoretically) rare nowadays, and fgets does work for both LF and CRLF.

The appropriate way to read text files in Cocoa/Objective-C is documented in Apple's String programming guide. The section for reading and writing files should be just what you're after. PS: What's a "line"? Two sections of a string separated by "\n"? Or "\r"? Or "\r\n"? Or maybe you're actually after paragraphs? The previously mentioned guide also includes a section on splitting a string into lines or paragraphs. (This section is called "Paragraphs and Line Breaks", and is linked to in the left-hand-side menu of the page I pointed to above. Unfortunately this site doesn't allow me to post more than one URL as I'm not a trustworthy user yet.)

To paraphrase Knuth: premature optimisation is the root of all evil. Don't simply assume that "reading the whole file into memory" is slow. Have you benchmarked it? Do you know that it actually reads the whole file into memory? Maybe it simply returns a proxy object and keeps reading behind the scenes as you consume the string? (Disclaimer: I have no idea if NSString actually does this. It conceivably could.) The point is: first go with the documented way of doing things. Then, if benchmarks show that this doesn't have the performance you desire, optimise.

Note
Rectangle 27 0

Objective C: Reading a file line by line?


// read everything from text
NSString* fileContents = 
      [NSString stringWithContentsOfFile:fileRoot 
       encoding:NSUTF8StringEncoding error:nil];

// first, separate by new line
NSArray* allLinedStrings = 
      [fileContents componentsSeparatedByCharactersInSet:
      [NSCharacterSet newlineCharacterSet]];

// then break down even further 
NSString* strsInOneLine = 
      [allLinedStrings objectAtIndex:0];

// choose whatever input identity you have decided. in this case ;
NSArray* singleStrs = 
      [currentPointString componentsSeparatedByCharactersInSet:
      [NSCharacterSet characterSetWithCharactersInString:@";"]];
NSString* filePath = @""//file path...
NSString* fileRoot = [[NSBundle mainBundle] 
               pathForResource:filePath ofType:@"txt"];

There you have it.

This is no response to the question. The question was to read a file line by line to reduce memory usage

This will work for general reading a String from Text. If you would like to read longer text (large size of text), then use the method that other people here were mentioned such as buffered (reserve the size of the text in memory space).

i have a 70 mb file , using this code to read file doesn't hep me it increases memory linearly. can any one help me?

Note
Rectangle 27 0

Objective C: Reading a file line by line?


FILE* file = fopen("path to my file", "r");

size_t length;
char *cLine = fgetln(file,&length);

while (length>0) {
    char str[length+1];
    strncpy(str, cLine, length);
    str[length] = '\0';

    NSString *line = [NSString stringWithFormat:@"%s",str];        
    % Do what you want here.

    cLine = fgetln(file,&length);
}

A lot of these answers are long chunks of code or they read in the entire file. I like to use the c methods for this very task.

Note that fgetln will not keep your newline character. Also, We +1 the length of the str because we want to make space for the NULL termination.

Note
Rectangle 27 0

Objective C: Reading a file line by line?


Mac OS X is Unix, Objective-C is C superset, so you can just use old-school fopen and fgets from <stdio.h>. It's guaranteed to work.

[NSString stringWithUTF8String:buf] will convert C string to NSString. There are also methods for creating strings in other encodings and creating without copying.

[copying anonymous comment] fgets will include the '\n' character, so you may want to strip that off before converting the string.

Note
Rectangle 27 0

Objective C: Reading a file line by line?


#import "DDFileReader.h"

@interface NSData (DDAdditions)

- (NSRange) rangeOfData_dd:(NSData *)dataToFind;

@end

@implementation NSData (DDAdditions)

- (NSRange) rangeOfData_dd:(NSData *)dataToFind {

    const void * bytes = [self bytes];
    NSUInteger length = [self length];

    const void * searchBytes = [dataToFind bytes];
    NSUInteger searchLength = [dataToFind length];
    NSUInteger searchIndex = 0;

    NSRange foundRange = {NSNotFound, searchLength};
    for (NSUInteger index = 0; index < length; index++) {
        if (((char *)bytes)[index] == ((char *)searchBytes)[searchIndex]) {
            //the current character matches
            if (foundRange.location == NSNotFound) {
                foundRange.location = index;
            }
            searchIndex++;
            if (searchIndex >= searchLength) { return foundRange; }
        } else {
            searchIndex = 0;
            foundRange.location = NSNotFound;
        }
    }
    return foundRange;
}

@end

@implementation DDFileReader
@synthesize lineDelimiter, chunkSize;

- (id) initWithFilePath:(NSString *)aPath {
    if (self = [super init]) {
        fileHandle = [NSFileHandle fileHandleForReadingAtPath:aPath];
        if (fileHandle == nil) {
            [self release]; return nil;
        }

        lineDelimiter = [[NSString alloc] initWithString:@"\n"];
        [fileHandle retain];
        filePath = [aPath retain];
        currentOffset = 0ULL;
        chunkSize = 10;
        [fileHandle seekToEndOfFile];
        totalFileLength = [fileHandle offsetInFile];
        //we don't need to seek back, since readLine will do that.
    }
    return self;
}

- (void) dealloc {
    [fileHandle closeFile];
    [fileHandle release], fileHandle = nil;
    [filePath release], filePath = nil;
    [lineDelimiter release], lineDelimiter = nil;
    currentOffset = 0ULL;
    [super dealloc];
}

- (NSString *) readLine {
    if (currentOffset >= totalFileLength) { return nil; }

    NSData * newLineData = [lineDelimiter dataUsingEncoding:NSUTF8StringEncoding];
    [fileHandle seekToFileOffset:currentOffset];
    NSMutableData * currentData = [[NSMutableData alloc] init];
    BOOL shouldReadMore = YES;

    NSAutoreleasePool * readPool = [[NSAutoreleasePool alloc] init];
    while (shouldReadMore) {
        if (currentOffset >= totalFileLength) { break; }
        NSData * chunk = [fileHandle readDataOfLength:chunkSize];
        NSRange newLineRange = [chunk rangeOfData_dd:newLineData];
        if (newLineRange.location != NSNotFound) {

            //include the length so we can include the delimiter in the string
            chunk = [chunk subdataWithRange:NSMakeRange(0, newLineRange.location+[newLineData length])];
            shouldReadMore = NO;
        }
        [currentData appendData:chunk];
        currentOffset += [chunk length];
    }
    [readPool release];

    NSString * line = [[NSString alloc] initWithData:currentData encoding:NSUTF8StringEncoding];
    [currentData release];
    return [line autorelease];
}

- (NSString *) readTrimmedLine {
    return [[self readLine] stringByTrimmingCharactersInSet:[NSCharacterSet whitespaceAndNewlineCharacterSet]];
}

#if NS_BLOCKS_AVAILABLE
- (void) enumerateLinesUsingBlock:(void(^)(NSString*, BOOL*))block {
  NSString * line = nil;
  BOOL stop = NO;
  while (stop == NO && (line = [self readLine])) {
    block(line, &stop);
  }
}
#endif

@end
@interface DDFileReader : NSObject {
    NSString * filePath;

    NSFileHandle * fileHandle;
    unsigned long long currentOffset;
    unsigned long long totalFileLength;

    NSString * lineDelimiter;
    NSUInteger chunkSize;
}

@property (nonatomic, copy) NSString * lineDelimiter;
@property (nonatomic) NSUInteger chunkSize;

- (id) initWithFilePath:(NSString *)aPath;

- (NSString *) readLine;
- (NSString *) readTrimmedLine;

#if NS_BLOCKS_AVAILABLE
- (void) enumerateLinesUsingBlock:(void(^)(NSString*, BOOL *))block;
#endif

@end
DDFileReader * reader = [[DDFileReader alloc] initWithFilePath:pathToMyFile];
[reader enumerateLinesUsingBlock:^(NSString * line, BOOL * stop) {
  NSLog(@"read line: %@", line);
}];
[reader release];

The class DDFileReader that enables this is the following:

The class was done by Dave DeLong

To read a file line by line (also for extreme big files) can be done by the following functions:

Note
Rectangle 27 0

Objective C: Reading a file line by line?


You can use NSInputStream which has a basic implementation for file streams. You can read bytes into a buffer (read:maxLength: method). You have to scan the buffer for newlines yourself.

Note