Bulk storing in SQLite while parsing in iOS

Discussion in 'iOS Programming' started by ashwinr87, Oct 31, 2011.

  1. ashwinr87 macrumors member

    Joined:
    Mar 9, 2011
    #1
    In my iOS application, I am parsing an XML from a SOAP based webservice and I am storing it into the SQLITE database. The problem that I am facing it is that, it is very slow. It takes about 18 seconds for about 310 rows of data.

    Here is what is a sample of my XML -

    Code:
    <return>
    <batteryID>1234</batteryID>
    <batteryVersionNum>1</batteryVersionNum>
    <conceptCode>abc</conceptCode>
    <conceptDescription>abc</conceptDescription>
    <effectiveEndTime>2010-11-23</effectiveEndTime>
    <effectiveStartTime>2010-11-23</effectiveStartTime>
    </return>
    <return>
    <batteryID>2345</batteryID>
    <batteryVersionNum>1</batteryVersionNum>
    <conceptCode>bac</conceptCode>
    <conceptDescription>bac</conceptDescription>
    <effectiveEndTime>2010-11-23</effectiveEndTime>
    <effectiveStartTime>2010-11-23</effectiveStartTime>
    </return>
    I use NSXML parser for parsing the XML. For every return tag() I encounter, I create a new instance of my entity -

    Code:
    - (void)parser:(NSXMLParser *)parser didStartElement:(NSString *)elementName
      namespaceURI:(NSString *)namespaceURI qualifiedName:(NSString *)qualifiedName
        attributes:(NSDictionary *)attributeDict
    {
        if ([elementName isEqualToString:@"return"])
        {
            // Blank lab panel object
            objLabPanel = [NSEntityDescription insertNewObjectForEntityForName:@"LabPanels" inManagedObjectContext:managedObjectContext];
            mainElement = elementName;
        }
    }
    Once I encounter a return end element (), I save the object to the database -
    Code:
    - (void)parser:(NSXMLParser *)parser didEndElement:(NSString *)elementName
      namespaceURI:(NSString *)namespaceURI qualifiedName:(NSString *)qName {
    
        if ([elementName isEqualToString:@"return"])
        {
            [objPatient addLabPanelsObject:objLabPanel];
    
            // Save
            NSError *error = nil;
            BOOL saveObj = FALSE;
            saveObj = [managedObjectContext save:&error];
    
            if (saveObj == FALSE)
            {
                NSLog (@"Error: %@", error);
            }
        }
        else if ([elementName isEqualToString:@"batteryID"] && [mainElement    isEqualToString:@"return"])
        {
        objLabPanel.labPanelBatteryId = elementValue;
        }
        // getting other values from the xml
        . 
        .
        .
        }
    Hence for every return tag I encounter, I do a save to the database and so for my 310 or so records, I would be saving it 310 times. So by doing a [managedObjectContext save:&error]; , I guess that it opens the database, saves the data and closes the database and hence it takes a lot of time. Am I right in thinking this way?

    Is there a way I can save all the objLabPanel's to an array and then do a bulk insert to the database i.e. bulk insert the entire array of objLabPanel's into the SQLITE database at once?

    It would be great if someone could help me out with this.
     
  2. dejo Moderator

    dejo

    Staff Member

    Joined:
    Sep 2, 2004
    Location:
    The Centennial State
  3. ashwinr87 thread starter macrumors member

    Joined:
    Mar 9, 2011
    #3
    it is similar but I feel that, I am splitting the problem into more detail with this question... while there I am looking for alternate solutions... but if you feel that it is the same, I can go ahead and remove the previous question..

     
  4. ChristianJapan macrumors 601

    ChristianJapan

    Joined:
    May 10, 2010
    Location:
    日本
    #4
    As suggested in the other thread use instruments to find where you have the time consuming parts in your code.its good invested time.

    Don't have no experience with the managed objects but would assume that you could make a kind of singleton class / factory as data model root and create first the XML file in memory. Once all data read and in memory buildup the data structures and dependencies you write the singleton and it's child tree to the storage. But again more guess, never done myself.
     
  5. ashwinr87 thread starter macrumors member

    Joined:
    Mar 9, 2011
    #5
    Thanks for the reply.. I found out where I was going wrong.. every time I encountered the
    Code:
    </return>
    i used to
    Code:
    [managedObjectContext save:&error];
    and that was taking the time.. so I changed it so that I do that only at the end and it made my code much faster.. it takes now only around 2 seconds to parse and store around 310 records...

     
  6. ChristianJapan macrumors 601

    ChristianJapan

    Joined:
    May 10, 2010
    Location:
    日本
    #6
    Problem soved by design decision: Very good ... Also good as it will save you the open/close and transactional overhead of writing to DB by using only one translation/commit insert of 310. One question you need to answer yourself: can you risk that your data might not be written complete in case something happen ? can the data from web easy reproduce/reloaded ?
     
  7. ashwinr87 thread starter macrumors member

    Joined:
    Mar 9, 2011
    #7
    yes.. that is the same question I did ask myself.. but I am not able to figure out the exact solution for it.. one thing is maybe I can store every 500 records or so.. but am not sure...
    any idea regarding how to solve this?

     

Share This Page