Hi Harald,

Can you please try load the data in batch mode setting the "uselazyAdd" 
parameter in your connect string to true ie:

        - VirtuosoRepository virtRepo = new 
VirtuosoRepository("jdbc:virtuoso://localhost:1111,dba,dba", "dba", "dba", 
"http://test.com/";);
        + VirtuosoRepository virtRepo = new 
VirtuosoRepository("jdbc:virtuoso://localhost:1111,dba,dba", "dba", "dba", 
"http://test.com/";, true);

This parameter enabling the batch optimized loading of data as detailed in the 
Javadocs for the VirtuosoRepository method at:

        http://docs.openlinksw.com/sesame/

Development are continuing to look into the cause of the "too many open 
statements" error when performing a single load, as that should work even if 
slower than the batch mode ...

Best Regards
Hugh Williams
Professional Services
OpenLink Software, Inc.      //              http://www.openlinksw.com/
Weblog   -- http://www.openlinksw.com/blogs/
LinkedIn -- http://www.linkedin.com/company/openlink-software/
Twitter  -- http://twitter.com/OpenLink
Google+  -- http://plus.google.com/100570109519069333827/
Facebook -- http://www.facebook.com/OpenLinkSoftware
Universal Data Access, Integration, and Management Technology Providers

On 6 Mar 2013, at 14:12, Harald Beck <[email protected]> wrote:

> Hi Hugh,
> 
> What I'm using is (essentially) something like this:
> 
> import org.openrdf.model.Statement;
> import org.openrdf.repository.Repository;
> import org.openrdf.repository.RepositoryConnection;
> import org.openrdf.repository.RepositoryException;
> import org.openrdf.rio.RDFHandler;
> import org.openrdf.rio.RDFHandlerException;
> import org.openrdf.rio.helpers.RDFHandlerBase;
> import virtuoso.sesame2.driver.VirtuosoRepository;
> 
> 
> public class Test {
> 
>    public void export(Repository projectRepo) throws RepositoryException, 
> RDFHandlerException {
> 
>        RepositoryConnection projectConn = projectRepo.getConnection();
> 
>        try {
>            VirtuosoRepository virtRepo = new 
> VirtuosoRepository("jdbc:virtuoso://localhost:1111,dba,dba", "dba", "dba", 
> "http://test.com/";);
>            final RepositoryConnection virtConn = virtRepo.getConnection();    
>         
> 
>            RDFHandler virtHandler = new RDFHandlerBase() {
>                @Override
>                public void handleStatement(Statement st) throws 
> RDFHandlerException {
>                    try {
>                        virtConn.add(st);
>                    } catch (RepositoryException ex) {
>                        throw new RDFHandlerException(ex);
>                    }
>                }
>            };
> 
>            projectConn.export(virtHandler);
>            virtConn.commit();
>            virtConn.close();
> 
>        } finally {
>            projectConn.close();
>        }
>    }
> 
> }
> 
> In the meantime, I could narrow down the problem: I can also trigger the 
> error by using the following method:
> 
> void export(Collection<Statement> statements) throws RepositoryException {
>    VirtuosoRepository virtRepo = new 
> VirtuosoRepository("jdbc:virtuoso://localhost:1111,dba,dba", "dba", "dba", 
> "http://test.com/";);
>    RepositoryConnection virtConn = virtRepo.getConnection();    
>    for (Statement s : statements) {
>       virtConn.add(s);
>    }    
>    virtConn.commit();
>    virtConn.close();
> }
> 
> However, if I use 
> 
> virtConn.add(statements)
> 
> instead of the above for-loop, the problem does NOT occur. So, there seems to 
> be a problem with too many .add calls.
> 
> Most of the time, it happens already before 10.000 triples. Committing 
> manually during the loop, or setting auto commit to true does not help.
> 
> By the way, which performance should I expect to have? With the above method, 
> I can only insert between 250 and 650 triples/sec (inserting seems to slow 
> down on larger files), while DB.DBA.TTLP_MT achieves around 8.000-13.000 
> triples/sec on my machine (ubuntu 12.04.1, 64bit, 8 GB RAM, Intel Core 
> i7-2670QM CPU @ 2.20GHz × 8). 
> 
> Best,
> Harald
> 
> 
> 
> 
> 
> 
> ----- Original Message -----
> From: "Hugh Williams" <[email protected]>
> To: "Harald Beck" <[email protected]>
> Cc: [email protected]
> Sent: Tuesday, March 5, 2013 1:55:25 PM
> Subject: Re: [Virtuoso-users]  Too many open statements
> 
> Hi Harald,
> 
> This issue should be resolved unless you are hitting another use case which 
> forces the condition to occur, thus are you able to provide a simple 
> reproducible test case we can setup in-house to see this issue ?
> 
> Best Regards
> Hugh Williams
> Professional Services
> OpenLink Software, Inc.      //              http://www.openlinksw.com/
> Weblog   -- http://www.openlinksw.com/blogs/
> LinkedIn -- http://www.linkedin.com/company/openlink-software/
> Twitter  -- http://twitter.com/OpenLink
> Google+  -- http://plus.google.com/100570109519069333827/
> Facebook -- http://www.facebook.com/OpenLinkSoftware
> Universal Data Access, Integration, and Management Technology Providers
> 
> On 4 Mar 2013, at 13:14, Harald Beck <[email protected]> wrote:
> 
>> Hi,
>> 
>> similarly as Andrea in 
>> http://www.mail-archive.com/[email protected]/msg05185.html
>> 
>> I get the
>> 
>> virtuoso.jdbc3.VirtuosoException: SR491: Too many open statements
>> 
>> In the last attempt, it happened after inserting 17838 triples (into 
>> Virtuoso Opensource 6.1.6).
>> 
>> I'm using the current virtjdbc3 driver (Build 3.62).
>> 
>> Any suggestions what to inspect?
>> 
>> Best,
>> Harald
>> 
>> -- 
>> Harald Beck
>> Software Developer
>> 
>> Semantic Web Company GmbH
>> Mariahilfer Straße 70 / 8
>> A - 1070 Vienna, Austria
>> Tel +43 1 402 12 35
>> Fax +43 1 402 12 35 - 22
>> 
>> http://www.semantic-web.at
>> http://blog.semantic-web.at
>> http://poolparty.biz
>> 
>> LOD2 - Creating Knowledge out of Interlinked Data - http://lod2.eu/
>> 
>> ------------------------------------------------------------------------------
>> Everyone hates slow websites. So do we.
>> Make your web apps faster with AppDynamics
>> Download AppDynamics Lite for free today:
>> http://p.sf.net/sfu/appdyn_d2d_feb
>> _______________________________________________
>> Virtuoso-users mailing list
>> [email protected]
>> https://lists.sourceforge.net/lists/listinfo/virtuoso-users
> 
> 
> -- 
> Harald Beck
> Software Developer
> 
> Semantic Web Company GmbH
> Mariahilfer Straße 70 / 8
> A - 1070 Vienna, Austria
> Tel +43 1 402 12 35
> Fax +43 1 402 12 35 - 22
> 
> http://www.semantic-web.at
> http://blog.semantic-web.at
> http://poolparty.biz
> 
> LOD2 - Creating Knowledge out of Interlinked Data - http://lod2.eu/
> 
> ------------------------------------------------------------------------------
> Symantec Endpoint Protection 12 positioned as A LEADER in The Forrester  
> Wave(TM): Endpoint Security, Q1 2013 and "remains a good choice" in the  
> endpoint security space. For insight on selecting the right partner to 
> tackle endpoint security challenges, access the full report. 
> http://p.sf.net/sfu/symantec-dev2dev
> _______________________________________________
> Virtuoso-users mailing list
> [email protected]
> https://lists.sourceforge.net/lists/listinfo/virtuoso-users

Attachment: smime.p7s
Description: S/MIME cryptographic signature

------------------------------------------------------------------------------
Everyone hates slow websites. So do we.
Make your web apps faster with AppDynamics
Download AppDynamics Lite for free today:
http://p.sf.net/sfu/appdyn_d2d_mar
_______________________________________________
Virtuoso-users mailing list
[email protected]
https://lists.sourceforge.net/lists/listinfo/virtuoso-users

Reply via email to