I just discovered SBJSON <http://www.superloopy.io/json-framework/> which handles chunked JSON data and it's actually working quite well. I also just discovered CBLJSONReader but I failed to get YAJL to build, due to "LLVM 6.0 Garbage collection is no longer allowed"
On Saturday, April 11, 2015 at 8:25:17 AM UTC-7, Jared McFarland wrote: > > I am getting line delimited JSON from Mixpanel.com's raw export API and > I'm looking for a more efficient way to handle this task than what I'm > currently doing. > > For reference here is the API: > https://mixpanel.com/docs/api-documentation/exporting-raw-data-you-inserted-into-mixpanel > And if you aren't already familiar with LDJSON: > http://en.wikipedia.org/wiki/Line_Delimited_JSON > > Right now I'm just waiting for the entire response to complete and then > transforming the data into an NSArray of NSDictionaries like so... > > NSError *jsonError; > > NSString *jsonString = [[NSString alloc] initWithData:data encoding: > NSUTF8StringEncoding]; // data is returned by my NSURLSessionDataTask's > completion handler > > > > NSMutableArray *resultsArray = [[jsonString > componentsSeparatedByString:@"\n"] mutableCopy]; > > [resultsArray removeObjectAtIndex:[resultsArray count]-1]; // remove > empty last object > > > > NSMutableArray *jsonArray = [NSMutableArray array]; > > > > for (NSString *result in resultsArray) { > > NSDictionary *jsonEvent = [NSJSONSerialization > JSONObjectWithData:[result > dataUsingEncoding:NSUTF8StringEncoding] options:0 error:&jsonError]; > > [jsonArray addObject:jsonEvent]; > > } > > Then I just take jsonArray and... > > BOOL transaction = [database inTransaction:^BOOL{ > > for (NSDictionary *event in jsonArray) { > > CBLDocument *document = [database createDocument]; > > NSError *documentError; > > NSMutableDictionary *mutableEvent = [event mutableCopy]; > > [mutableEvent setObject:kMPCBLDocumentTypeEvent forKey: > kMPCBLDocumentKeyType]; > > [document putProperties:mutableEvent error:&documentError]; > > if (documentError) { > > return NO; > > } > > } > > return YES; > > }]; > > > However, the whole point of LDSON is that you don't have to wait for the > entire response, so I'm trying to figure out the best way to go about > parsing the data as it streams in, and probably calling inTransaction: on > batches of 1000 (or so) event objects. These responses can be 100,000's of > documents, or even millions in rare cases. > > > The NSURLSessionDataDelegate protocol has the didReceiveData: method but > I'm struggling to properly parse the results and wondering if there is > already some well established pattern for serializing LDSON as it is > received. > > > Any input would be greatly appreciated! > -- You received this message because you are subscribed to the Google Groups "Couchbase Mobile" group. To unsubscribe from this group and stop receiving emails from it, send an email to [email protected]. To view this discussion on the web visit https://groups.google.com/d/msgid/mobile-couchbase/27219bb3-425b-4c42-8f16-d7661d6cff76%40googlegroups.com. For more options, visit https://groups.google.com/d/optout.
