Thanks for the info Noam, Our typical use falls in the small artifact range so we can most likely use MySQL. Luckily we also set Artifactory up on a fairly beefy machine with 2 quad core CPUs, 4GB memory and lots of disk space so we can probably get away with running the DB on the same machine for now. This will enable us to switch to an external DB faster than trying to get new hardware with 64bit O.S. We can put the new hardware on an upgrade plan for the future.
Thanks for your help, Ken _____________________________________________________ Ken Pacileo UnitedHealth Group | Continuous Integration Services From: Noam Y. Tenne [mailto:[email protected]] Sent: Wednesday, October 03, 2012 9:55 AM To: [email protected] Subject: Re: [Artifactory-users] Metadata file and artifact errors during backup I'd classify large artifacts as 500MB+. Whether the DB and Artifactory should be hosted on the same machine depends on the strength of the machine and the resources required by both applications; the added response time latency of communications between 2 servers may be minuscule compared to latency caused by an over-burdened machine. On Mon, Oct 1, 2012 at 1:44 PM, Pacileo, Ken <[email protected]> wrote: Hi Noam, Thanks for your reply and suggestions. I was afraid you were going to say we need to upgrade the DB. ☺ With our current constraints we may need to try just upgrading to MySQL for now and delay upgrading to 64bit JVM until next year. When you refer to physically large artifacts, what size range are you referring to? 500MB, 1GB, etc.? With either MySQL or Oracle, is the preference to have the DB on the same server Artifactory is running on rather than locate the DB on a different server? Our company typically has the Oracle DBs on separate servers which would increase response times. I would rather not jeopardize builds due to timeouts because DB requests are not responding fast enough. If the preference is to have the DB on the same server then we may need to stay with MySQL. I still need to check if we can use Postgre though. Thanks and regards, Ken _____________________________________________________ Ken Pacileo UnitedHealth Group | Continuous Integration Services From: Noam Y. Tenne [mailto:[email protected]] Sent: Sunday, September 30, 2012 2:51 AM To: [email protected] Subject: Re: [Artifactory-users] Metadata file and artifact errors during backup Ken, The issue here is most likely Derby DB. You can buy some time and a bit of room to breath by providing the Artifactory process with more memory (and should probably run on a 64bit JVM); but if you expect usage to grow, this certainly won't hold and I would recommend migrating to an external DB. The type of DB to use normally depends on the number and typical size of the artifacts your instance handles; we normally recommend filesystem-MySQL for when you've got a large number of small artifacts, and a full-on DB like Oracle or Postgre for handling physically large artifacts. If you do decide to migrate, just make sure you've got a good set of backups to import from and schedule some downtime for the migration process. On Sat, Sep 29, 2012 at 9:20 PM, Pacileo, Ken <[email protected]> wrote: Hi Noam, Thank you for your reply. We're on the default Derby DB. Our JVM is Oracle JDK 6u32 with these params: -XX:PermSize=128m -XX:MaxPermSize=128m -XX:SurvivorRatio=8 -XX:NewSize=512m -XX:MaxNewSize=512m -XX:+UseParallelGC -XX:-UseConcMarkSweepGC -Xms768m -Xmx1536m -Xss256k Is the database a potential problem since we are on the default Derby DB? If so and we need to switch to one of the supported databases would MySQL be adequate? The "data" directory is 75.2GB now. If we do need to migrate, what potential issues do we need to be on the lookout for? Thanks and regards, Ken _____________________________________________________ Ken Pacileo UnitedHealth Group | Continuous Integration Services From: Noam Y. Tenne [mailto:[email protected]] Sent: Saturday, September 29, 2012 5:12 AM To: [email protected] Subject: Re: [Artifactory-users] Metadata file and artifact errors during backup Hi Ken, Which type of database type do you use? Can you please send us the JVM params with which you start Artifactory? On Tue, Sep 18, 2012 at 4:02 PM, Pacileo, Ken <[email protected]> wrote: My apologies, I sent the original email from the wrong mailbox. Please reply to this email instead. Thanks and regards, Ken Pacileo -----Original Message----- From: Continuous Integration Services Sent: Tuesday, September 18, 2012 8:59 AM To: [email protected] Subject: [Artifactory-users] Metadata file and artifact errors during backup Hi, Occasionally we receive emails from the backup process that the backup completed with errors. We seem to get two types of errors: 1) a failure to read a maven-metadata.xml file for an artifactid's directory or the artifactId's version directory or 2) a failure to export an artifact since it is non-accessible. These errors do not stop the backup from running but we do need to take action on each error and remove the offending entry from the UI or the next backup will have the same error(s). For the first case, when we navigate the UI to the maven-metadata.xml file there is a message where the file's contents should be that the file is corrupt. We remove the file and the next backup usually succeeds. For the second case, when we navigate to the actual artifact, the size of the file is shown as -1. We then delete the artifact and it's corresponding pom file. In the artifactory.log file there is a message stating "2012-09-18 07:28:11,839 [http-4580-148] [WARN ] (o.a.j.c.v.BLOBInDataStore:97) - Could not get size for 40569b3846dce7455cc1d5c7259f5d2d77d478f4" which appears when we click on the artifact name in the UI before we delete it. I've scrubbed the repository name (the errors are always in our local Snapshot repository), and the groupId and artifactId names in both the email data and log file data below. This is the contents of the email we get from the backup process. Failed to export /repositories/XXX-Snapshots/abc/def/ghi/ABC_Foo/9.0.0-SNAPSHOT/ABC_Foo-9 .0.0-20120917.121706-7.jar since it is non-accessible.: Data stream for /repositories/XXX-Snapshots/abc/def/ghi/ABC_Foo/9.0.0-SNAPSHOT/ABC_Foo-9 .0.0-20120917.121706-7.jar not found! Due to Failed to retrieve file node's /repositories/XXX-Snapshots/abc/def/ghi/ABC_Foo/9.0.0-SNAPSHOT/ABC_Foo-9 .0.0-20120917.121706-7.jar data stream. Failed to export metadata for '/repositories/XXX-Snapshots/abc/def/ghi/ABC_Bar'.: Failed to read JCR DB for data 'maven-metadata.xml' of /repositories/XXX-Snapshots/abc/def/ghi/ABC_Bar Failed to export metadata for '/repositories/XXX-Snapshots/abc/def/ghi/ABC_Bar/7.0.4-SNAPSHOT'.: Failed to read JCR DB for data 'maven-metadata.xml' of /repositories/XXX-Snapshots/abc/def/ghi/ABC_Bar/7.0.4-SNAPSHOT Failed to export metadata for '/repositories/XXX-Snapshots/abc/def/ghi/ABC_Bar/7.0.8-SNAPSHOT'.: Failed to read JCR DB for data 'maven-metadata.xml' of /repositories/XXX-Snapshots/abc/def/ghi/ABC_Bar/7.0.8-SNAPSHOT Failed to export /repositories/XXX-Snapshots/abc/def/ghi/ABC_Bar/9.0.0-SNAPSHOT/ABC_Bar-9 .0.0-20120917.121359-7.jar since it is non-accessible.: Data stream for /repositories/XXX-Snapshots/abc/def/ghi/ABC_Bar/9.0.0-SNAPSHOT/ABC_Bar-9 .0.0-20120917.121359-7.jar not found! Due to Failed to retrieve file node's /repositories/XXX-Snapshots/abc/def/ghi/ABC_Bar/9.0.0-SNAPSHOT/ABC_Bar-9 .0.0-20120917.121359-7.jar data stream. This is the relevant section of the log file showing the errors when the backup process ran and again when we click on each object in the UI. 2012-09-18 05:32:30,706 [art-exec-2704] [ERROR] (o.a.j.f.JcrFsItem :788) - Failed to export metadata for '/repositories/XXX-Snapshots/abc/def/ghi/ABC_Bar'.: Failed to read JCR DB for data 'maven-metadata.xml' of /repositories/XXX-Snapshots/abc/def/ghi/ABC_Bar 2012-09-18 05:32:31,300 [art-exec-2704] [ERROR] (o.a.j.f.JcrFsItem :788) - Failed to export metadata for '/repositories/XXX-Snapshots/abc/def/ghi/ABC_Bar/7.0.4-SNAPSHOT'.: Failed to read JCR DB for data 'maven-metadata.xml' of /repositories/XXX-Snapshots/abc/def/ghi/ABC_Bar/7.0.4-SNAPSHOT 2012-09-18 05:32:31,847 [art-exec-2704] [ERROR] (o.a.j.f.JcrFsItem :788) - Failed to export metadata for '/repositories/XXX-Snapshots/abc/def/ghi/ABC_Bar/7.0.8-SNAPSHOT'.: Failed to read JCR DB for data 'maven-metadata.xml' of /repositories/XXX-Snapshots/abc/def/ghi/ABC_Bar/7.0.8-SNAPSHOT 2012-09-18 07:03:18,639 [http-4580-150] [ERROR] (o.a.w.w.p.b.t.t.m.MetadataPanel:152) - Error while retrieving selected metadata 'maven-metadata.xml': Failed to read JCR DB for data 'maven-metadata.xml' of /repositories/XXX-Snapshots/abc/def/ghi/ABC_Bar 2012-09-18 07:08:51,426 [http-4580-78] [ERROR] (o.a.w.w.p.b.t.t.m.MetadataPanel:152) - Error while retrieving selected metadata 'maven-metadata.xml': Failed to read JCR DB for data 'maven-metadata.xml' of /repositories/XXX-Snapshots/abc/def/ghi/ABC_Bar/7.0.4-SNAPSHOT 2012-09-18 07:25:09,695 [http-4580-150] [ERROR] (o.a.w.w.p.b.t.t.m.MetadataPanel:152) - Error while retrieving selected metadata 'maven-metadata.xml': Failed to read JCR DB for data 'maven-metadata.xml' of /repositories/XXX-Snapshots/abc/def/ghi/ABC_Bar/7.0.8-SNAPSHOT We are using Artifactory v2.6.1 with Tomcat 6.0.26 and Oracle JDK 6u32 on a Windows 2003 R2 SP2 server with 3.25GB RAM and 2 Intel Xeon CPUs. We've been upgrading our version of Artifactory from 2.3.2 to 2.4.x to 2.6.1 and have seen these errors in all the versions. Has anyone else seen similar types of backup errors? Any idea what is the cause and what the fix is? Thanks and regards, Ken Pacileo This e-mail, including attachments, may include confidential and/or proprietary information, and may be used only by the person or entity to which it is addressed. If the reader of this e-mail is not the intended recipient or his or her authorized agent, the reader is hereby notified that any dissemination, distribution or copying of this e-mail is prohibited. If you have received this e-mail in error, please notify the sender by replying to this message and delete this e-mail immediately. ------------------------------------------------------------------------ ------ Live Security Virtual Conference Exclusive live event will cover all the ways today's security and threat landscape has changed and how IT managers can respond. Discussions will include endpoint security, mobile security and the latest in malware threats. http://www.accelacomm.com/jaw/sfrnl04242012/114/50122263/ _______________________________________________ Artifactory-users mailing list [email protected] https://lists.sourceforge.net/lists/listinfo/artifactory-users This e-mail, including attachments, may include confidential and/or proprietary information, and may be used only by the person or entity to which it is addressed. If the reader of this e-mail is not the intended recipient or his or her authorized agent, the reader is hereby notified that any dissemination, distribution or copying of this e-mail is prohibited. If you have received this e-mail in error, please notify the sender by replying to this message and delete this e-mail immediately. ------------------------------------------------------------------------------ Live Security Virtual Conference Exclusive live event will cover all the ways today's security and threat landscape has changed and how IT managers can respond. Discussions will include endpoint security, mobile security and the latest in malware threats. http://www.accelacomm.com/jaw/sfrnl04242012/114/50122263/ _______________________________________________ Artifactory-users mailing list [email protected] https://lists.sourceforge.net/lists/listinfo/artifactory-users This e-mail, including attachments, may include confidential and/or proprietary information, and may be used only by the person or entity to which it is addressed. If the reader of this e-mail is not the intended recipient or his or her authorized agent, the reader is hereby notified that any dissemination, distribution or copying of this e-mail is prohibited. If you have received this e-mail in error, please notify the sender by replying to this message and delete this e-mail immediately. ------------------------------------------------------------------------------ How fast is your code? 3 out of 4 devs don\\\'t know how their code performs in production. Find out how slow your code is with AppDynamics Lite. http://ad.doubleclick.net/clk;262219672;13503038;z? http://info.appdynamics.com/FreeJavaPerformanceDownload.html _______________________________________________ Artifactory-users mailing list [email protected] https://lists.sourceforge.net/lists/listinfo/artifactory-users This e-mail, including attachments, may include confidential and/or proprietary information, and may be used only by the person or entity to which it is addressed. If the reader of this e-mail is not the intended recipient or his or her authorized agent, the reader is hereby notified that any dissemination, distribution or copying of this e-mail is prohibited. If you have received this e-mail in error, please notify the sender by replying to this message and delete this e-mail immediately. ------------------------------------------------------------------------------ Got visibility? Most devs has no idea what their production app looks like. Find out how fast your code is with AppDynamics Lite. http://ad.doubleclick.net/clk;262219671;13503038;y? http://info.appdynamics.com/FreeJavaPerformanceDownload.html _______________________________________________ Artifactory-users mailing list [email protected] https://lists.sourceforge.net/lists/listinfo/artifactory-users This e-mail, including attachments, may include confidential and/or proprietary information, and may be used only by the person or entity to which it is addressed. If the reader of this e-mail is not the intended recipient or his or her authorized agent, the reader is hereby notified that any dissemination, distribution or copying of this e-mail is prohibited. If you have received this e-mail in error, please notify the sender by replying to this message and delete this e-mail immediately. ------------------------------------------------------------------------------ Don't let slow site performance ruin your business. Deploy New Relic APM Deploy New Relic app performance management and know exactly what is happening inside your Ruby, Python, PHP, Java, and .NET app Try New Relic at no cost today and get our sweet Data Nerd shirt too! http://p.sf.net/sfu/newrelic-dev2dev _______________________________________________ Artifactory-users mailing list [email protected] https://lists.sourceforge.net/lists/listinfo/artifactory-users
