Ken,
The issue here is most likely Derby DB.
You can buy some time and a bit of room to breath by providing the
Artifactory process with more memory (and should probably run on a 64bit
JVM); but if you expect usage to grow, this certainly won't hold and I
would recommend migrating to an external DB.
The type of DB to use normally depends on the number and typical size of
the artifacts your instance handles; we normally recommend filesystem-MySQL
for when you've got a large number of small artifacts, and a full-on DB
like Oracle or Postgre for handling physically large artifacts.
If you do decide to migrate, just make sure you've got a good set of
backups to import from and schedule some downtime for the migration process.
On Sat, Sep 29, 2012 at 9:20 PM, Pacileo, Ken <[email protected]> wrote:
> Hi Noam,
>
> Thank you for your reply.
>
> We're on the default Derby DB.
>
> Our JVM is Oracle JDK 6u32 with these params:
> -XX:PermSize=128m
> -XX:MaxPermSize=128m
> -XX:SurvivorRatio=8
> -XX:NewSize=512m
> -XX:MaxNewSize=512m
> -XX:+UseParallelGC
> -XX:-UseConcMarkSweepGC
> -Xms768m
> -Xmx1536m
> -Xss256k
>
> Is the database a potential problem since we are on the default Derby DB?
> If so and we need to switch to one of the supported databases would MySQL
> be adequate? The "data" directory is 75.2GB now. If we do need to migrate,
> what potential issues do we need to be on the lookout for?
>
> Thanks and regards,
> Ken
>
> _____________________________________________________
> Ken Pacileo
> UnitedHealth Group | Continuous Integration Services
>
> From: Noam Y. Tenne [mailto:[email protected]]
> Sent: Saturday, September 29, 2012 5:12 AM
> To: [email protected]
> Subject: Re: [Artifactory-users] Metadata file and artifact errors
> duringbackup
>
> Hi Ken,
>
> Which type of database type do you use?
> Can you please send us the JVM params with which you start Artifactory?
>
> On Tue, Sep 18, 2012 at 4:02 PM, Pacileo, Ken <[email protected]> wrote:
> My apologies, I sent the original email from the wrong mailbox. Please
> reply to this email instead.
>
> Thanks and regards,
> Ken Pacileo
>
> -----Original Message-----
> From: Continuous Integration Services
> Sent: Tuesday, September 18, 2012 8:59 AM
> To: [email protected]
> Subject: [Artifactory-users] Metadata file and artifact errors during
> backup
>
> Hi,
>
> Occasionally we receive emails from the backup process that the backup
> completed with errors. We seem to get two types of errors: 1) a failure
> to read a maven-metadata.xml file for an artifactid's directory or the
> artifactId's version directory or 2) a failure to export an artifact
> since it is non-accessible.
>
> These errors do not stop the backup from running but we do need to take
> action on each error and remove the offending entry from the UI or the
> next backup will have the same error(s).
>
> For the first case, when we navigate the UI to the maven-metadata.xml
> file there is a message where the file's contents should be that the
> file is corrupt. We remove the file and the next backup usually
> succeeds.
>
> For the second case, when we navigate to the actual artifact, the size
> of the file is shown as -1. We then delete the artifact and it's
> corresponding pom file. In the artifactory.log file there is a message
> stating "2012-09-18 07:28:11,839 [http-4580-148] [WARN ]
> (o.a.j.c.v.BLOBInDataStore:97) - Could not get size for
> 40569b3846dce7455cc1d5c7259f5d2d77d478f4" which appears when we click on
> the artifact name in the UI before we delete it.
>
>
> I've scrubbed the repository name (the errors are always in our local
> Snapshot repository), and the groupId and artifactId names in both the
> email data and log file data below.
>
> This is the contents of the email we get from the backup process.
> Failed to export
> /repositories/XXX-Snapshots/abc/def/ghi/ABC_Foo/9.0.0-SNAPSHOT/ABC_Foo-9
> .0.0-20120917.121706-7.jar since it is non-accessible.: Data stream for
> /repositories/XXX-Snapshots/abc/def/ghi/ABC_Foo/9.0.0-SNAPSHOT/ABC_Foo-9
> .0.0-20120917.121706-7.jar not found! Due to Failed to retrieve file
> node's
> /repositories/XXX-Snapshots/abc/def/ghi/ABC_Foo/9.0.0-SNAPSHOT/ABC_Foo-9
> .0.0-20120917.121706-7.jar data stream.
> Failed to export metadata for
> '/repositories/XXX-Snapshots/abc/def/ghi/ABC_Bar'.: Failed to read JCR
> DB for data 'maven-metadata.xml' of
> /repositories/XXX-Snapshots/abc/def/ghi/ABC_Bar
> Failed to export metadata for
> '/repositories/XXX-Snapshots/abc/def/ghi/ABC_Bar/7.0.4-SNAPSHOT'.:
> Failed to read JCR DB for data 'maven-metadata.xml' of
> /repositories/XXX-Snapshots/abc/def/ghi/ABC_Bar/7.0.4-SNAPSHOT
> Failed to export metadata for
> '/repositories/XXX-Snapshots/abc/def/ghi/ABC_Bar/7.0.8-SNAPSHOT'.:
> Failed to read JCR DB for data 'maven-metadata.xml' of
> /repositories/XXX-Snapshots/abc/def/ghi/ABC_Bar/7.0.8-SNAPSHOT
> Failed to export
> /repositories/XXX-Snapshots/abc/def/ghi/ABC_Bar/9.0.0-SNAPSHOT/ABC_Bar-9
> .0.0-20120917.121359-7.jar since it is non-accessible.: Data stream for
> /repositories/XXX-Snapshots/abc/def/ghi/ABC_Bar/9.0.0-SNAPSHOT/ABC_Bar-9
> .0.0-20120917.121359-7.jar not found! Due to Failed to retrieve file
> node's
> /repositories/XXX-Snapshots/abc/def/ghi/ABC_Bar/9.0.0-SNAPSHOT/ABC_Bar-9
> .0.0-20120917.121359-7.jar data stream.
>
> This is the relevant section of the log file showing the errors when the
> backup process ran and again when we click on each object in the UI.
> 2012-09-18 05:32:30,706 [art-exec-2704] [ERROR] (o.a.j.f.JcrFsItem
> :788) - Failed to export metadata for
> '/repositories/XXX-Snapshots/abc/def/ghi/ABC_Bar'.: Failed to read JCR
> DB for data 'maven-metadata.xml' of
> /repositories/XXX-Snapshots/abc/def/ghi/ABC_Bar
> 2012-09-18 05:32:31,300 [art-exec-2704] [ERROR] (o.a.j.f.JcrFsItem
> :788) - Failed to export metadata for
> '/repositories/XXX-Snapshots/abc/def/ghi/ABC_Bar/7.0.4-SNAPSHOT'.:
> Failed to read JCR DB for data 'maven-metadata.xml' of
> /repositories/XXX-Snapshots/abc/def/ghi/ABC_Bar/7.0.4-SNAPSHOT
> 2012-09-18 05:32:31,847 [art-exec-2704] [ERROR] (o.a.j.f.JcrFsItem
> :788) - Failed to export metadata for
> '/repositories/XXX-Snapshots/abc/def/ghi/ABC_Bar/7.0.8-SNAPSHOT'.:
> Failed to read JCR DB for data 'maven-metadata.xml' of
> /repositories/XXX-Snapshots/abc/def/ghi/ABC_Bar/7.0.8-SNAPSHOT
> 2012-09-18 07:03:18,639 [http-4580-150] [ERROR]
> (o.a.w.w.p.b.t.t.m.MetadataPanel:152) - Error while retrieving selected
> metadata 'maven-metadata.xml': Failed to read JCR DB for data
> 'maven-metadata.xml' of /repositories/XXX-Snapshots/abc/def/ghi/ABC_Bar
> 2012-09-18 07:08:51,426 [http-4580-78] [ERROR]
> (o.a.w.w.p.b.t.t.m.MetadataPanel:152) - Error while retrieving selected
> metadata 'maven-metadata.xml': Failed to read JCR DB for data
> 'maven-metadata.xml' of
> /repositories/XXX-Snapshots/abc/def/ghi/ABC_Bar/7.0.4-SNAPSHOT
> 2012-09-18 07:25:09,695 [http-4580-150] [ERROR]
> (o.a.w.w.p.b.t.t.m.MetadataPanel:152) - Error while retrieving selected
> metadata 'maven-metadata.xml': Failed to read JCR DB for data
> 'maven-metadata.xml' of
> /repositories/XXX-Snapshots/abc/def/ghi/ABC_Bar/7.0.8-SNAPSHOT
>
> We are using Artifactory v2.6.1 with Tomcat 6.0.26 and Oracle JDK 6u32
> on a Windows 2003 R2 SP2 server with 3.25GB RAM and 2 Intel Xeon CPUs.
> We've been upgrading our version of Artifactory from 2.3.2 to 2.4.x to
> 2.6.1 and have seen these errors in all the versions.
>
> Has anyone else seen similar types of backup errors? Any idea what is
> the cause and what the fix is?
>
> Thanks and regards,
> Ken Pacileo
>
>
>
> This e-mail, including attachments, may include confidential and/or
> proprietary information, and may be used only by the person or entity to
> which it is addressed. If the reader of this e-mail is not the intended
> recipient or his or her authorized agent, the reader is hereby notified
> that any dissemination, distribution or copying of this e-mail is
> prohibited. If you have received this e-mail in error, please notify the
> sender by replying to this message and delete this e-mail immediately.
>
>
> ------------------------------------------------------------------------
> ------
> Live Security Virtual Conference
> Exclusive live event will cover all the ways today's security and threat
> landscape has changed and how IT managers can respond. Discussions will
> include endpoint security, mobile security and the latest in malware
> threats. http://www.accelacomm.com/jaw/sfrnl04242012/114/50122263/
> _______________________________________________
> Artifactory-users mailing list
> [email protected]
> https://lists.sourceforge.net/lists/listinfo/artifactory-users
>
> This e-mail, including attachments, may include confidential and/or
> proprietary information, and may be used only by the person or entity
> to which it is addressed. If the reader of this e-mail is not the intended
> recipient or his or her authorized agent, the reader is hereby notified
> that any dissemination, distribution or copying of this e-mail is
> prohibited. If you have received this e-mail in error, please notify the
> sender by replying to this message and delete this e-mail immediately.
>
>
>
> ------------------------------------------------------------------------------
> Live Security Virtual Conference
> Exclusive live event will cover all the ways today's security and
> threat landscape has changed and how IT managers can respond. Discussions
> will include endpoint security, mobile security and the latest in malware
> threats. http://www.accelacomm.com/jaw/sfrnl04242012/114/50122263/
> _______________________________________________
> Artifactory-users mailing list
> [email protected]
> https://lists.sourceforge.net/lists/listinfo/artifactory-users
>
>
> This e-mail, including attachments, may include confidential and/or
> proprietary information, and may be used only by the person or entity
> to which it is addressed. If the reader of this e-mail is not the intended
> recipient or his or her authorized agent, the reader is hereby notified
> that any dissemination, distribution or copying of this e-mail is
> prohibited. If you have received this e-mail in error, please notify the
> sender by replying to this message and delete this e-mail immediately.
>
>
>
> ------------------------------------------------------------------------------
> How fast is your code?
> 3 out of 4 devs don\\\'t know how their code performs in production.
> Find out how slow your code is with AppDynamics Lite.
> http://ad.doubleclick.net/clk;262219672;13503038;z?
> http://info.appdynamics.com/FreeJavaPerformanceDownload.html
> _______________________________________________
> Artifactory-users mailing list
> [email protected]
> https://lists.sourceforge.net/lists/listinfo/artifactory-users
>
------------------------------------------------------------------------------
Everyone hates slow websites. So do we.
Make your web apps faster with AppDynamics
Download AppDynamics Lite for free today:
http://ad.doubleclick.net/clk;258768047;13503038;j?
http://info.appdynamics.com/FreeJavaPerformanceDownload.html
_______________________________________________
Artifactory-users mailing list
[email protected]
https://lists.sourceforge.net/lists/listinfo/artifactory-users
