Re: [MTT devel] MTT Database and Reporter Upgrade **Action Required**
I think probably the easiest thing to do is just mv reporter.php and submit.php so that mtt clients and individuals browsing will not be able to access it. On Aug 26, 2007, at 10:58 PM, Josh Hursey wrote: Jeff and Ethan, How do I shutdown the MTT server? Do I just log into milliways as mtt, or can I disable it from my user? Cheers, -- Josh Begin forwarded message: From: Josh Hursey Date: August 24, 2007 1:37:18 PM EDT To: General user list for the MPI Testing Tool Subject: [MTT users] MTT Database and Reporter Upgrade **Action Required** Reply-To: General user list for the MPI Testing Tool Short Version: -- The MTT development group is rolling out newly optimized web frontend and backend database. As a result we will be taking down the MTT site at IU Monday, August 27 from 8 am to Noon US eastern time. During this time you will not be able to submit data to the MTT database. Therefore you need to disable any runs that will report during this time or your client will fail with unable to connect to server messages. This change does not affect the client configurations, so MTT users do *not* need to update their clients at this time. Longer Version: --- The MTT development team has been working diligently on server side optimizations over the past few months. This work involved major changes to the database schema, web reporter, and web submit components of the server. We want to roll out the new server side optimizations on Monday, Aug. 27. Given the extensive nature of the improvements the MTT server will need to be taken down for a few hours for this upgrade to take place. We are planning on taking down the MTT server at 8 am and we hope to have it back by Noon US Eastern time. MTT users that would normally submit results during this time range will need to disable their runs, or they will see server error messages during this outage. This upgrade does not require any client changes, so outside of the down time contributors need not change or upgrade their MTT installations. Below are a few rough performance numbers illustrating the difference between the old and new server versions as seen by the reporter. Summary report: 24 hours, all orgs 87 sec - old version 6 sec - new version Summary report: 24 hours, org = 'iu' 37 sec - old 4 sec - new Summary report: Past 3 days, all orgs 138 sec - old 9 sec - new Summary report: Past 3 days, org = 'iu' 49 sec - old 11 sec - new Summary report: Past 2 weeks, all orgs 863 sec - old 34 sec - new Summary report: Past 2 weeks, org = 'iu' 878 sec - old 12 sec - new Summary report: Past 1 month, all org 1395 sec - old 158 sec - new Summary report: Past 1 month, org = 'iu' 1069 sec - old 39 sec - new Summary report: (2007-06-18 - 2007-06-19), all org 484 sec - old 5 sec - new Summary report: (2007-06-18 - 2007-06-19), org = 'iu' 479 sec - old 2 sec - new ___ mtt-users mailing list mtt-us...@open-mpi.org http://www.open-mpi.org/mailman/listinfo.cgi/mtt-users ___ mtt-devel mailing list mtt-de...@open-mpi.org http://www.open-mpi.org/mailman/listinfo.cgi/mtt-devel -- Jeff Squyres Cisco Systems
[MTT devel] Upgrade cron question
Are the following two scripts needed anymore? monitor_disk_usage.pl prune_db.pl They both rely on the old db schema. If they are needed then we need to update them. I renamed 'prune_db.pl' to 'DEACTIVATE-prune_db.pl' so it is not accidently run, as I don't know if it would mangle the new db or not. I did this locally in: /l/osl/www/www.open-mpi.org/mtt/cron Whos account are the cron scripts running currently? Cheers, Josh
Re: [MTT devel] MTT Database and Reporter Upgrade **Action Required**
Yeah I was unsure if I had the proper permissions to move the file. Apparently I do :) The upgrade is finished now. I just ran a simple MTT on the Odin machine at IU and it seemed to have gone through well. Should we announce to the larger list or did you all want to do some testing first? -- josh On Aug 27, 2007, at 7:48 AM, Jeff Squyres wrote: I think probably the easiest thing to do is just mv reporter.php and submit.php so that mtt clients and individuals browsing will not be able to access it. On Aug 26, 2007, at 10:58 PM, Josh Hursey wrote: Jeff and Ethan, How do I shutdown the MTT server? Do I just log into milliways as mtt, or can I disable it from my user? Cheers, -- Josh Begin forwarded message: From: Josh Hursey Date: August 24, 2007 1:37:18 PM EDT To: General user list for the MPI Testing Tool Subject: [MTT users] MTT Database and Reporter Upgrade **Action Required** Reply-To: General user list for the MPI Testing Tool Short Version: -- The MTT development group is rolling out newly optimized web frontend and backend database. As a result we will be taking down the MTT site at IU Monday, August 27 from 8 am to Noon US eastern time. During this time you will not be able to submit data to the MTT database. Therefore you need to disable any runs that will report during this time or your client will fail with unable to connect to server messages. This change does not affect the client configurations, so MTT users do *not* need to update their clients at this time. Longer Version: --- The MTT development team has been working diligently on server side optimizations over the past few months. This work involved major changes to the database schema, web reporter, and web submit components of the server. We want to roll out the new server side optimizations on Monday, Aug. 27. Given the extensive nature of the improvements the MTT server will need to be taken down for a few hours for this upgrade to take place. We are planning on taking down the MTT server at 8 am and we hope to have it back by Noon US Eastern time. MTT users that would normally submit results during this time range will need to disable their runs, or they will see server error messages during this outage. This upgrade does not require any client changes, so outside of the down time contributors need not change or upgrade their MTT installations. Below are a few rough performance numbers illustrating the difference between the old and new server versions as seen by the reporter. Summary report: 24 hours, all orgs 87 sec - old version 6 sec - new version Summary report: 24 hours, org = 'iu' 37 sec - old 4 sec - new Summary report: Past 3 days, all orgs 138 sec - old 9 sec - new Summary report: Past 3 days, org = 'iu' 49 sec - old 11 sec - new Summary report: Past 2 weeks, all orgs 863 sec - old 34 sec - new Summary report: Past 2 weeks, org = 'iu' 878 sec - old 12 sec - new Summary report: Past 1 month, all org 1395 sec - old 158 sec - new Summary report: Past 1 month, org = 'iu' 1069 sec - old 39 sec - new Summary report: (2007-06-18 - 2007-06-19), all org 484 sec - old 5 sec - new Summary report: (2007-06-18 - 2007-06-19), org = 'iu' 479 sec - old 2 sec - new ___ mtt-users mailing list mtt-us...@open-mpi.org http://www.open-mpi.org/mailman/listinfo.cgi/mtt-users ___ mtt-devel mailing list mtt-de...@open-mpi.org http://www.open-mpi.org/mailman/listinfo.cgi/mtt-devel -- Jeff Squyres Cisco Systems ___ mtt-devel mailing list mtt-de...@open-mpi.org http://www.open-mpi.org/mailman/listinfo.cgi/mtt-devel
Re: [MTT devel] MTT Database and Reporter Upgrade **Action Required**
Let's do some simple submits and ensure that we can query for them properly. I'm guessing that everything will be fine, but let's be thorough... I'll kick off a simple round of get/install/get/build/run for trivial tests now. On Aug 27, 2007, at 9:28 AM, Josh Hursey wrote: Yeah I was unsure if I had the proper permissions to move the file. Apparently I do :) The upgrade is finished now. I just ran a simple MTT on the Odin machine at IU and it seemed to have gone through well. Should we announce to the larger list or did you all want to do some testing first? -- josh On Aug 27, 2007, at 7:48 AM, Jeff Squyres wrote: I think probably the easiest thing to do is just mv reporter.php and submit.php so that mtt clients and individuals browsing will not be able to access it. On Aug 26, 2007, at 10:58 PM, Josh Hursey wrote: Jeff and Ethan, How do I shutdown the MTT server? Do I just log into milliways as mtt, or can I disable it from my user? Cheers, -- Josh Begin forwarded message: From: Josh Hursey Date: August 24, 2007 1:37:18 PM EDT To: General user list for the MPI Testing Tool Subject: [MTT users] MTT Database and Reporter Upgrade **Action Required** Reply-To: General user list for the MPI Testing Tool Short Version: -- The MTT development group is rolling out newly optimized web frontend and backend database. As a result we will be taking down the MTT site at IU Monday, August 27 from 8 am to Noon US eastern time. During this time you will not be able to submit data to the MTT database. Therefore you need to disable any runs that will report during this time or your client will fail with unable to connect to server messages. This change does not affect the client configurations, so MTT users do *not* need to update their clients at this time. Longer Version: --- The MTT development team has been working diligently on server side optimizations over the past few months. This work involved major changes to the database schema, web reporter, and web submit components of the server. We want to roll out the new server side optimizations on Monday, Aug. 27. Given the extensive nature of the improvements the MTT server will need to be taken down for a few hours for this upgrade to take place. We are planning on taking down the MTT server at 8 am and we hope to have it back by Noon US Eastern time. MTT users that would normally submit results during this time range will need to disable their runs, or they will see server error messages during this outage. This upgrade does not require any client changes, so outside of the down time contributors need not change or upgrade their MTT installations. Below are a few rough performance numbers illustrating the difference between the old and new server versions as seen by the reporter. Summary report: 24 hours, all orgs 87 sec - old version 6 sec - new version Summary report: 24 hours, org = 'iu' 37 sec - old 4 sec - new Summary report: Past 3 days, all orgs 138 sec - old 9 sec - new Summary report: Past 3 days, org = 'iu' 49 sec - old 11 sec - new Summary report: Past 2 weeks, all orgs 863 sec - old 34 sec - new Summary report: Past 2 weeks, org = 'iu' 878 sec - old 12 sec - new Summary report: Past 1 month, all org 1395 sec - old 158 sec - new Summary report: Past 1 month, org = 'iu' 1069 sec - old 39 sec - new Summary report: (2007-06-18 - 2007-06-19), all org 484 sec - old 5 sec - new Summary report: (2007-06-18 - 2007-06-19), org = 'iu' 479 sec - old 2 sec - new ___ mtt-users mailing list mtt-us...@open-mpi.org http://www.open-mpi.org/mailman/listinfo.cgi/mtt-users ___ mtt-devel mailing list mtt-de...@open-mpi.org http://www.open-mpi.org/mailman/listinfo.cgi/mtt-devel -- Jeff Squyres Cisco Systems ___ mtt-devel mailing list mtt-de...@open-mpi.org http://www.open-mpi.org/mailman/listinfo.cgi/mtt-devel ___ mtt-devel mailing list mtt-de...@open-mpi.org http://www.open-mpi.org/mailman/listinfo.cgi/mtt-devel -- Jeff Squyres Cisco Systems
Re: [MTT devel] MTT Database and Reporter Upgrade **Action Required**
Ok, it's looking good so far (I accidentally submitted one intel mpi result as a non "trial" -- oops). What's "Filter tracked failures" in the preferences window? The link on "tracked" goes to a 404. On Aug 27, 2007, at 9:48 AM, Jeff Squyres wrote: Let's do some simple submits and ensure that we can query for them properly. I'm guessing that everything will be fine, but let's be thorough... I'll kick off a simple round of get/install/get/build/run for trivial tests now. On Aug 27, 2007, at 9:28 AM, Josh Hursey wrote: Yeah I was unsure if I had the proper permissions to move the file. Apparently I do :) The upgrade is finished now. I just ran a simple MTT on the Odin machine at IU and it seemed to have gone through well. Should we announce to the larger list or did you all want to do some testing first? -- josh On Aug 27, 2007, at 7:48 AM, Jeff Squyres wrote: I think probably the easiest thing to do is just mv reporter.php and submit.php so that mtt clients and individuals browsing will not be able to access it. On Aug 26, 2007, at 10:58 PM, Josh Hursey wrote: Jeff and Ethan, How do I shutdown the MTT server? Do I just log into milliways as mtt, or can I disable it from my user? Cheers, -- Josh Begin forwarded message: From: Josh Hursey Date: August 24, 2007 1:37:18 PM EDT To: General user list for the MPI Testing Tool Subject: [MTT users] MTT Database and Reporter Upgrade **Action Required** Reply-To: General user list for the MPI Testing Tool Short Version: -- The MTT development group is rolling out newly optimized web frontend and backend database. As a result we will be taking down the MTT site at IU Monday, August 27 from 8 am to Noon US eastern time. During this time you will not be able to submit data to the MTT database. Therefore you need to disable any runs that will report during this time or your client will fail with unable to connect to server messages. This change does not affect the client configurations, so MTT users do *not* need to update their clients at this time. Longer Version: --- The MTT development team has been working diligently on server side optimizations over the past few months. This work involved major changes to the database schema, web reporter, and web submit components of the server. We want to roll out the new server side optimizations on Monday, Aug. 27. Given the extensive nature of the improvements the MTT server will need to be taken down for a few hours for this upgrade to take place. We are planning on taking down the MTT server at 8 am and we hope to have it back by Noon US Eastern time. MTT users that would normally submit results during this time range will need to disable their runs, or they will see server error messages during this outage. This upgrade does not require any client changes, so outside of the down time contributors need not change or upgrade their MTT installations. Below are a few rough performance numbers illustrating the difference between the old and new server versions as seen by the reporter. Summary report: 24 hours, all orgs 87 sec - old version 6 sec - new version Summary report: 24 hours, org = 'iu' 37 sec - old 4 sec - new Summary report: Past 3 days, all orgs 138 sec - old 9 sec - new Summary report: Past 3 days, org = 'iu' 49 sec - old 11 sec - new Summary report: Past 2 weeks, all orgs 863 sec - old 34 sec - new Summary report: Past 2 weeks, org = 'iu' 878 sec - old 12 sec - new Summary report: Past 1 month, all org 1395 sec - old 158 sec - new Summary report: Past 1 month, org = 'iu' 1069 sec - old 39 sec - new Summary report: (2007-06-18 - 2007-06-19), all org 484 sec - old 5 sec - new Summary report: (2007-06-18 - 2007-06-19), org = 'iu' 479 sec - old 2 sec - new ___ mtt-users mailing list mtt-us...@open-mpi.org http://www.open-mpi.org/mailman/listinfo.cgi/mtt-users ___ mtt-devel mailing list mtt-de...@open-mpi.org http://www.open-mpi.org/mailman/listinfo.cgi/mtt-devel -- Jeff Squyres Cisco Systems ___ mtt-devel mailing list mtt-de...@open-mpi.org http://www.open-mpi.org/mailman/listinfo.cgi/mtt-devel ___ mtt-devel mailing list mtt-de...@open-mpi.org http://www.open-mpi.org/mailman/listinfo.cgi/mtt-devel -- Jeff Squyres Cisco Systems ___ mtt-devel mailing list mtt-de...@open-mpi.org http://www.open-mpi.org/mailman/listinfo.cgi/mtt-devel -- Jeff Squyres Cisco Systems
Re: [MTT devel] MTT Database and Reporter Upgrade **Action Required**
I think this was something that Ethan was working with on the trunk. I don't think it works yet. We should probably remove it as an option. I'll see if I can do that easily. -- Josh On Aug 27, 2007, at 10:03 AM, Jeff Squyres wrote: Ok, it's looking good so far (I accidentally submitted one intel mpi result as a non "trial" -- oops). What's "Filter tracked failures" in the preferences window? The link on "tracked" goes to a 404. On Aug 27, 2007, at 9:48 AM, Jeff Squyres wrote: Let's do some simple submits and ensure that we can query for them properly. I'm guessing that everything will be fine, but let's be thorough... I'll kick off a simple round of get/install/get/build/run for trivial tests now. On Aug 27, 2007, at 9:28 AM, Josh Hursey wrote: Yeah I was unsure if I had the proper permissions to move the file. Apparently I do :) The upgrade is finished now. I just ran a simple MTT on the Odin machine at IU and it seemed to have gone through well. Should we announce to the larger list or did you all want to do some testing first? -- josh On Aug 27, 2007, at 7:48 AM, Jeff Squyres wrote: I think probably the easiest thing to do is just mv reporter.php and submit.php so that mtt clients and individuals browsing will not be able to access it. On Aug 26, 2007, at 10:58 PM, Josh Hursey wrote: Jeff and Ethan, How do I shutdown the MTT server? Do I just log into milliways as mtt, or can I disable it from my user? Cheers, -- Josh Begin forwarded message: From: Josh Hursey Date: August 24, 2007 1:37:18 PM EDT To: General user list for the MPI Testing Tool Subject: [MTT users] MTT Database and Reporter Upgrade **Action Required** Reply-To: General user list for the MPI Testing Tool Short Version: -- The MTT development group is rolling out newly optimized web frontend and backend database. As a result we will be taking down the MTT site at IU Monday, August 27 from 8 am to Noon US eastern time. During this time you will not be able to submit data to the MTT database. Therefore you need to disable any runs that will report during this time or your client will fail with unable to connect to server messages. This change does not affect the client configurations, so MTT users do *not* need to update their clients at this time. Longer Version: --- The MTT development team has been working diligently on server side optimizations over the past few months. This work involved major changes to the database schema, web reporter, and web submit components of the server. We want to roll out the new server side optimizations on Monday, Aug. 27. Given the extensive nature of the improvements the MTT server will need to be taken down for a few hours for this upgrade to take place. We are planning on taking down the MTT server at 8 am and we hope to have it back by Noon US Eastern time. MTT users that would normally submit results during this time range will need to disable their runs, or they will see server error messages during this outage. This upgrade does not require any client changes, so outside of the down time contributors need not change or upgrade their MTT installations. Below are a few rough performance numbers illustrating the difference between the old and new server versions as seen by the reporter. Summary report: 24 hours, all orgs 87 sec - old version 6 sec - new version Summary report: 24 hours, org = 'iu' 37 sec - old 4 sec - new Summary report: Past 3 days, all orgs 138 sec - old 9 sec - new Summary report: Past 3 days, org = 'iu' 49 sec - old 11 sec - new Summary report: Past 2 weeks, all orgs 863 sec - old 34 sec - new Summary report: Past 2 weeks, org = 'iu' 878 sec - old 12 sec - new Summary report: Past 1 month, all org 1395 sec - old 158 sec - new Summary report: Past 1 month, org = 'iu' 1069 sec - old 39 sec - new Summary report: (2007-06-18 - 2007-06-19), all org 484 sec - old 5 sec - new Summary report: (2007-06-18 - 2007-06-19), org = 'iu' 479 sec - old 2 sec - new ___ mtt-users mailing list mtt-us...@open-mpi.org http://www.open-mpi.org/mailman/listinfo.cgi/mtt-users ___ mtt-devel mailing list mtt-de...@open-mpi.org http://www.open-mpi.org/mailman/listinfo.cgi/mtt-devel -- Jeff Squyres Cisco Systems ___ mtt-devel mailing list mtt-de...@open-mpi.org http://www.open-mpi.org/mailman/listinfo.cgi/mtt-devel ___ mtt-devel mailing list mtt-de...@open-mpi.org http://www.open-mpi.org/mailman/listinfo.cgi/mtt-devel -- Jeff Squyres Cisco Systems ___ mtt-devel mailing list mtt-de...@open-mpi.org http://www.open-mpi.org/mailman/listinfo.cgi/mtt-devel -- Jeff Squyres Cisco Systems ___
Re: [MTT devel] MTT Database and Reporter Upgrade **Action Required**
Fixed. On Aug 27, 2007, at 10:03 AM, Jeff Squyres wrote: Ok, it's looking good so far (I accidentally submitted one intel mpi result as a non "trial" -- oops). What's "Filter tracked failures" in the preferences window? The link on "tracked" goes to a 404. On Aug 27, 2007, at 9:48 AM, Jeff Squyres wrote: Let's do some simple submits and ensure that we can query for them properly. I'm guessing that everything will be fine, but let's be thorough... I'll kick off a simple round of get/install/get/build/run for trivial tests now. On Aug 27, 2007, at 9:28 AM, Josh Hursey wrote: Yeah I was unsure if I had the proper permissions to move the file. Apparently I do :) The upgrade is finished now. I just ran a simple MTT on the Odin machine at IU and it seemed to have gone through well. Should we announce to the larger list or did you all want to do some testing first? -- josh On Aug 27, 2007, at 7:48 AM, Jeff Squyres wrote: I think probably the easiest thing to do is just mv reporter.php and submit.php so that mtt clients and individuals browsing will not be able to access it. On Aug 26, 2007, at 10:58 PM, Josh Hursey wrote: Jeff and Ethan, How do I shutdown the MTT server? Do I just log into milliways as mtt, or can I disable it from my user? Cheers, -- Josh Begin forwarded message: From: Josh Hursey Date: August 24, 2007 1:37:18 PM EDT To: General user list for the MPI Testing Tool Subject: [MTT users] MTT Database and Reporter Upgrade **Action Required** Reply-To: General user list for the MPI Testing Tool Short Version: -- The MTT development group is rolling out newly optimized web frontend and backend database. As a result we will be taking down the MTT site at IU Monday, August 27 from 8 am to Noon US eastern time. During this time you will not be able to submit data to the MTT database. Therefore you need to disable any runs that will report during this time or your client will fail with unable to connect to server messages. This change does not affect the client configurations, so MTT users do *not* need to update their clients at this time. Longer Version: --- The MTT development team has been working diligently on server side optimizations over the past few months. This work involved major changes to the database schema, web reporter, and web submit components of the server. We want to roll out the new server side optimizations on Monday, Aug. 27. Given the extensive nature of the improvements the MTT server will need to be taken down for a few hours for this upgrade to take place. We are planning on taking down the MTT server at 8 am and we hope to have it back by Noon US Eastern time. MTT users that would normally submit results during this time range will need to disable their runs, or they will see server error messages during this outage. This upgrade does not require any client changes, so outside of the down time contributors need not change or upgrade their MTT installations. Below are a few rough performance numbers illustrating the difference between the old and new server versions as seen by the reporter. Summary report: 24 hours, all orgs 87 sec - old version 6 sec - new version Summary report: 24 hours, org = 'iu' 37 sec - old 4 sec - new Summary report: Past 3 days, all orgs 138 sec - old 9 sec - new Summary report: Past 3 days, org = 'iu' 49 sec - old 11 sec - new Summary report: Past 2 weeks, all orgs 863 sec - old 34 sec - new Summary report: Past 2 weeks, org = 'iu' 878 sec - old 12 sec - new Summary report: Past 1 month, all org 1395 sec - old 158 sec - new Summary report: Past 1 month, org = 'iu' 1069 sec - old 39 sec - new Summary report: (2007-06-18 - 2007-06-19), all org 484 sec - old 5 sec - new Summary report: (2007-06-18 - 2007-06-19), org = 'iu' 479 sec - old 2 sec - new ___ mtt-users mailing list mtt-us...@open-mpi.org http://www.open-mpi.org/mailman/listinfo.cgi/mtt-users ___ mtt-devel mailing list mtt-de...@open-mpi.org http://www.open-mpi.org/mailman/listinfo.cgi/mtt-devel -- Jeff Squyres Cisco Systems ___ mtt-devel mailing list mtt-de...@open-mpi.org http://www.open-mpi.org/mailman/listinfo.cgi/mtt-devel ___ mtt-devel mailing list mtt-de...@open-mpi.org http://www.open-mpi.org/mailman/listinfo.cgi/mtt-devel -- Jeff Squyres Cisco Systems ___ mtt-devel mailing list mtt-de...@open-mpi.org http://www.open-mpi.org/mailman/listinfo.cgi/mtt-devel -- Jeff Squyres Cisco Systems ___ mtt-devel mailing list mtt-de...@open-mpi.org http://www.open-mpi.org/mailman/listinfo.cgi/mtt-devel
[MTT devel] Performance Graph Notes
Jeff asked me if I would talk to Rich and folks at IU on Friday about the performance graphs. Below are my notes from that meeting: - It would be really useful to 'zoom' into sections of the graph. Primarily restricting the x-axis (Message Size), but also having the ability to restrict the y-axis (time) - Calling the y-axis 'latency' is a bit misleading, maybe 'time' would be better. Minor issue. - Torsten mentioned that he was interested in seeing the other skampi data that we are throwing away. Namely the time-per-rank. And if available communicator size. - Torsten mentioned that he wants to add some non-blocking collective test that he is work on. I told him to contact Jeff on how to do this. - We need a well defined way to see what collective implementation was used. Meaning that there are N AlltoAll collective implementations in the 'tuned' component we need to know when looking at the graph which one of the N we are looking at for Open MPI. For other implementations we don't have so much control. - It is difficult to search in the reporter for queries like: -- Open MPI run with only tcp,sm,self ; Intel MPI (which is only tcp I believe) ; MPICH2 with tcp results from running the skampi Bcast benchmark. -- The reporter is designed to track a single MPI well for regression tracking. However when we need to compare multiple MPIs and each may need to be selected with a different type of query it is impossible/ hard to do. One solution I proposed was using the 'tagging' idea, but there might be some alternative UI features that we can develop to better support these types of queries. Tim P seemed interested/had some ideas on how to do this. - They really liked the ability to look at the HTML version of the raw data. They seemed frustrated that the popup window is reused when looking at multiple HTML versions of the raw data. They wanted this to be a static window that they could keep open so they could look at multiple variants of this data in small screens. That was about it. They thought it was good over all, but the above were suggestions on ways to make the representation more useful. -- Josh
Re: [MTT devel] Performance Graph Notes
A few more notes I forgot: - The performance graphs are sometimes placed side-by-side instead of stacked on top of one another. This shinks the x-axis, and is undesirable. They would prefer that the graphs be always stacked on top of one another. - They lamented the lack of the cherry picking feature since it is known to be broken in the new reporter. - They noticed that sometimes there is 'wasted space' in the graphs in both the x and y axis. They want the graph to be pushed to the edges of the graph so they can see the most detail in the results. I think that's really it this time. :) On Aug 27, 2007, at 10:33 AM, Josh Hursey wrote: Jeff asked me if I would talk to Rich and folks at IU on Friday about the performance graphs. Below are my notes from that meeting: - It would be really useful to 'zoom' into sections of the graph. Primarily restricting the x-axis (Message Size), but also having the ability to restrict the y-axis (time) - Calling the y-axis 'latency' is a bit misleading, maybe 'time' would be better. Minor issue. - Torsten mentioned that he was interested in seeing the other skampi data that we are throwing away. Namely the time-per-rank. And if available communicator size. - Torsten mentioned that he wants to add some non-blocking collective test that he is work on. I told him to contact Jeff on how to do this. - We need a well defined way to see what collective implementation was used. Meaning that there are N AlltoAll collective implementations in the 'tuned' component we need to know when looking at the graph which one of the N we are looking at for Open MPI. For other implementations we don't have so much control. - It is difficult to search in the reporter for queries like: -- Open MPI run with only tcp,sm,self ; Intel MPI (which is only tcp I believe) ; MPICH2 with tcp results from running the skampi Bcast benchmark. -- The reporter is designed to track a single MPI well for regression tracking. However when we need to compare multiple MPIs and each may need to be selected with a different type of query it is impossible/ hard to do. One solution I proposed was using the 'tagging' idea, but there might be some alternative UI features that we can develop to better support these types of queries. Tim P seemed interested/had some ideas on how to do this. - They really liked the ability to look at the HTML version of the raw data. They seemed frustrated that the popup window is reused when looking at multiple HTML versions of the raw data. They wanted this to be a static window that they could keep open so they could look at multiple variants of this data in small screens. That was about it. They thought it was good over all, but the above were suggestions on ways to make the representation more useful. -- Josh ___ mtt-devel mailing list mtt-de...@open-mpi.org http://www.open-mpi.org/mailman/listinfo.cgi/mtt-devel
Re: [MTT devel] MTT Database and Reporter Upgrade **Action Required**
Schweet. On Aug 27, 2007, at 10:17 AM, Josh Hursey wrote: Fixed. On Aug 27, 2007, at 10:03 AM, Jeff Squyres wrote: Ok, it's looking good so far (I accidentally submitted one intel mpi result as a non "trial" -- oops). What's "Filter tracked failures" in the preferences window? The link on "tracked" goes to a 404. On Aug 27, 2007, at 9:48 AM, Jeff Squyres wrote: Let's do some simple submits and ensure that we can query for them properly. I'm guessing that everything will be fine, but let's be thorough... I'll kick off a simple round of get/install/get/build/run for trivial tests now. On Aug 27, 2007, at 9:28 AM, Josh Hursey wrote: Yeah I was unsure if I had the proper permissions to move the file. Apparently I do :) The upgrade is finished now. I just ran a simple MTT on the Odin machine at IU and it seemed to have gone through well. Should we announce to the larger list or did you all want to do some testing first? -- josh On Aug 27, 2007, at 7:48 AM, Jeff Squyres wrote: I think probably the easiest thing to do is just mv reporter.php and submit.php so that mtt clients and individuals browsing will not be able to access it. On Aug 26, 2007, at 10:58 PM, Josh Hursey wrote: Jeff and Ethan, How do I shutdown the MTT server? Do I just log into milliways as mtt, or can I disable it from my user? Cheers, -- Josh Begin forwarded message: From: Josh Hursey Date: August 24, 2007 1:37:18 PM EDT To: General user list for the MPI Testing Tool Subject: [MTT users] MTT Database and Reporter Upgrade **Action Required** Reply-To: General user list for the MPI Testing Tool Short Version: -- The MTT development group is rolling out newly optimized web frontend and backend database. As a result we will be taking down the MTT site at IU Monday, August 27 from 8 am to Noon US eastern time. During this time you will not be able to submit data to the MTT database. Therefore you need to disable any runs that will report during this time or your client will fail with unable to connect to server messages. This change does not affect the client configurations, so MTT users do *not* need to update their clients at this time. Longer Version: --- The MTT development team has been working diligently on server side optimizations over the past few months. This work involved major changes to the database schema, web reporter, and web submit components of the server. We want to roll out the new server side optimizations on Monday, Aug. 27. Given the extensive nature of the improvements the MTT server will need to be taken down for a few hours for this upgrade to take place. We are planning on taking down the MTT server at 8 am and we hope to have it back by Noon US Eastern time. MTT users that would normally submit results during this time range will need to disable their runs, or they will see server error messages during this outage. This upgrade does not require any client changes, so outside of the down time contributors need not change or upgrade their MTT installations. Below are a few rough performance numbers illustrating the difference between the old and new server versions as seen by the reporter. Summary report: 24 hours, all orgs 87 sec - old version 6 sec - new version Summary report: 24 hours, org = 'iu' 37 sec - old 4 sec - new Summary report: Past 3 days, all orgs 138 sec - old 9 sec - new Summary report: Past 3 days, org = 'iu' 49 sec - old 11 sec - new Summary report: Past 2 weeks, all orgs 863 sec - old 34 sec - new Summary report: Past 2 weeks, org = 'iu' 878 sec - old 12 sec - new Summary report: Past 1 month, all org 1395 sec - old 158 sec - new Summary report: Past 1 month, org = 'iu' 1069 sec - old 39 sec - new Summary report: (2007-06-18 - 2007-06-19), all org 484 sec - old 5 sec - new Summary report: (2007-06-18 - 2007-06-19), org = 'iu' 479 sec - old 2 sec - new ___ mtt-users mailing list mtt-us...@open-mpi.org http://www.open-mpi.org/mailman/listinfo.cgi/mtt-users ___ mtt-devel mailing list mtt-de...@open-mpi.org http://www.open-mpi.org/mailman/listinfo.cgi/mtt-devel -- Jeff Squyres Cisco Systems ___ mtt-devel mailing list mtt-de...@open-mpi.org http://www.open-mpi.org/mailman/listinfo.cgi/mtt-devel ___ mtt-devel mailing list mtt-de...@open-mpi.org http://www.open-mpi.org/mailman/listinfo.cgi/mtt-devel -- Jeff Squyres Cisco Systems ___ mtt-devel mailing list mtt-de...@open-mpi.org http://www.open-mpi.org/mailman/listinfo.cgi/mtt-devel -- Jeff Squyres Cisco Systems ___ mtt-devel mailing list mtt-de...@open-mpi.org http://www.open-mpi.org/mailman/listinfo.cgi/mtt-d
Re: [MTT devel] MTT Database and Reporter Upgrade **Action Required**
BTW, I think we're good to go. Ethan? On Aug 27, 2007, at 10:17 AM, Josh Hursey wrote: Fixed. On Aug 27, 2007, at 10:03 AM, Jeff Squyres wrote: Ok, it's looking good so far (I accidentally submitted one intel mpi result as a non "trial" -- oops). What's "Filter tracked failures" in the preferences window? The link on "tracked" goes to a 404. On Aug 27, 2007, at 9:48 AM, Jeff Squyres wrote: Let's do some simple submits and ensure that we can query for them properly. I'm guessing that everything will be fine, but let's be thorough... I'll kick off a simple round of get/install/get/build/run for trivial tests now. On Aug 27, 2007, at 9:28 AM, Josh Hursey wrote: Yeah I was unsure if I had the proper permissions to move the file. Apparently I do :) The upgrade is finished now. I just ran a simple MTT on the Odin machine at IU and it seemed to have gone through well. Should we announce to the larger list or did you all want to do some testing first? -- josh On Aug 27, 2007, at 7:48 AM, Jeff Squyres wrote: I think probably the easiest thing to do is just mv reporter.php and submit.php so that mtt clients and individuals browsing will not be able to access it. On Aug 26, 2007, at 10:58 PM, Josh Hursey wrote: Jeff and Ethan, How do I shutdown the MTT server? Do I just log into milliways as mtt, or can I disable it from my user? Cheers, -- Josh Begin forwarded message: From: Josh Hursey Date: August 24, 2007 1:37:18 PM EDT To: General user list for the MPI Testing Tool Subject: [MTT users] MTT Database and Reporter Upgrade **Action Required** Reply-To: General user list for the MPI Testing Tool Short Version: -- The MTT development group is rolling out newly optimized web frontend and backend database. As a result we will be taking down the MTT site at IU Monday, August 27 from 8 am to Noon US eastern time. During this time you will not be able to submit data to the MTT database. Therefore you need to disable any runs that will report during this time or your client will fail with unable to connect to server messages. This change does not affect the client configurations, so MTT users do *not* need to update their clients at this time. Longer Version: --- The MTT development team has been working diligently on server side optimizations over the past few months. This work involved major changes to the database schema, web reporter, and web submit components of the server. We want to roll out the new server side optimizations on Monday, Aug. 27. Given the extensive nature of the improvements the MTT server will need to be taken down for a few hours for this upgrade to take place. We are planning on taking down the MTT server at 8 am and we hope to have it back by Noon US Eastern time. MTT users that would normally submit results during this time range will need to disable their runs, or they will see server error messages during this outage. This upgrade does not require any client changes, so outside of the down time contributors need not change or upgrade their MTT installations. Below are a few rough performance numbers illustrating the difference between the old and new server versions as seen by the reporter. Summary report: 24 hours, all orgs 87 sec - old version 6 sec - new version Summary report: 24 hours, org = 'iu' 37 sec - old 4 sec - new Summary report: Past 3 days, all orgs 138 sec - old 9 sec - new Summary report: Past 3 days, org = 'iu' 49 sec - old 11 sec - new Summary report: Past 2 weeks, all orgs 863 sec - old 34 sec - new Summary report: Past 2 weeks, org = 'iu' 878 sec - old 12 sec - new Summary report: Past 1 month, all org 1395 sec - old 158 sec - new Summary report: Past 1 month, org = 'iu' 1069 sec - old 39 sec - new Summary report: (2007-06-18 - 2007-06-19), all org 484 sec - old 5 sec - new Summary report: (2007-06-18 - 2007-06-19), org = 'iu' 479 sec - old 2 sec - new ___ mtt-users mailing list mtt-us...@open-mpi.org http://www.open-mpi.org/mailman/listinfo.cgi/mtt-users ___ mtt-devel mailing list mtt-de...@open-mpi.org http://www.open-mpi.org/mailman/listinfo.cgi/mtt-devel -- Jeff Squyres Cisco Systems ___ mtt-devel mailing list mtt-de...@open-mpi.org http://www.open-mpi.org/mailman/listinfo.cgi/mtt-devel ___ mtt-devel mailing list mtt-de...@open-mpi.org http://www.open-mpi.org/mailman/listinfo.cgi/mtt-devel -- Jeff Squyres Cisco Systems ___ mtt-devel mailing list mtt-de...@open-mpi.org http://www.open-mpi.org/mailman/listinfo.cgi/mtt-devel -- Jeff Squyres Cisco Systems ___ mtt-devel mailing list mtt-de...@open-mpi.org http://www.open-mpi
Re: [MTT devel] MTT Database and Reporter Upgrade **Action Required**
I say we are good to go since we are approaching Noon. We can work out the bugs as we go. What do you all think? -- Josh On Aug 27, 2007, at 10:50 AM, Jeff Squyres wrote: BTW, I think we're good to go. Ethan? On Aug 27, 2007, at 10:17 AM, Josh Hursey wrote: Fixed. On Aug 27, 2007, at 10:03 AM, Jeff Squyres wrote: Ok, it's looking good so far (I accidentally submitted one intel mpi result as a non "trial" -- oops). What's "Filter tracked failures" in the preferences window? The link on "tracked" goes to a 404. On Aug 27, 2007, at 9:48 AM, Jeff Squyres wrote: Let's do some simple submits and ensure that we can query for them properly. I'm guessing that everything will be fine, but let's be thorough... I'll kick off a simple round of get/install/get/build/run for trivial tests now. On Aug 27, 2007, at 9:28 AM, Josh Hursey wrote: Yeah I was unsure if I had the proper permissions to move the file. Apparently I do :) The upgrade is finished now. I just ran a simple MTT on the Odin machine at IU and it seemed to have gone through well. Should we announce to the larger list or did you all want to do some testing first? -- josh On Aug 27, 2007, at 7:48 AM, Jeff Squyres wrote: I think probably the easiest thing to do is just mv reporter.php and submit.php so that mtt clients and individuals browsing will not be able to access it. On Aug 26, 2007, at 10:58 PM, Josh Hursey wrote: Jeff and Ethan, How do I shutdown the MTT server? Do I just log into milliways as mtt, or can I disable it from my user? Cheers, -- Josh Begin forwarded message: From: Josh Hursey Date: August 24, 2007 1:37:18 PM EDT To: General user list for the MPI Testing Tool Subject: [MTT users] MTT Database and Reporter Upgrade **Action Required** Reply-To: General user list for the MPI Testing Tool Short Version: -- The MTT development group is rolling out newly optimized web frontend and backend database. As a result we will be taking down the MTT site at IU Monday, August 27 from 8 am to Noon US eastern time. During this time you will not be able to submit data to the MTT database. Therefore you need to disable any runs that will report during this time or your client will fail with unable to connect to server messages. This change does not affect the client configurations, so MTT users do *not* need to update their clients at this time. Longer Version: --- The MTT development team has been working diligently on server side optimizations over the past few months. This work involved major changes to the database schema, web reporter, and web submit components of the server. We want to roll out the new server side optimizations on Monday, Aug. 27. Given the extensive nature of the improvements the MTT server will need to be taken down for a few hours for this upgrade to take place. We are planning on taking down the MTT server at 8 am and we hope to have it back by Noon US Eastern time. MTT users that would normally submit results during this time range will need to disable their runs, or they will see server error messages during this outage. This upgrade does not require any client changes, so outside of the down time contributors need not change or upgrade their MTT installations. Below are a few rough performance numbers illustrating the difference between the old and new server versions as seen by the reporter. Summary report: 24 hours, all orgs 87 sec - old version 6 sec - new version Summary report: 24 hours, org = 'iu' 37 sec - old 4 sec - new Summary report: Past 3 days, all orgs 138 sec - old 9 sec - new Summary report: Past 3 days, org = 'iu' 49 sec - old 11 sec - new Summary report: Past 2 weeks, all orgs 863 sec - old 34 sec - new Summary report: Past 2 weeks, org = 'iu' 878 sec - old 12 sec - new Summary report: Past 1 month, all org 1395 sec - old 158 sec - new Summary report: Past 1 month, org = 'iu' 1069 sec - old 39 sec - new Summary report: (2007-06-18 - 2007-06-19), all org 484 sec - old 5 sec - new Summary report: (2007-06-18 - 2007-06-19), org = 'iu' 479 sec - old 2 sec - new ___ mtt-users mailing list mtt-us...@open-mpi.org http://www.open-mpi.org/mailman/listinfo.cgi/mtt-users ___ mtt-devel mailing list mtt-de...@open-mpi.org http://www.open-mpi.org/mailman/listinfo.cgi/mtt-devel -- Jeff Squyres Cisco Systems ___ mtt-devel mailing list mtt-de...@open-mpi.org http://www.open-mpi.org/mailman/listinfo.cgi/mtt-devel ___ mtt-devel mailing list mtt-de...@open-mpi.org http://www.open-mpi.org/mailman/listinfo.cgi/mtt-devel -- Jeff Squyres Cisco Systems ___ mtt-devel mailing list mtt-de...@open-mpi.org http://www.open-m
Re: [MTT devel] MTT Database and Reporter Upgrade **Action Required**
I see a bunch of submits in the db -- including some from IBM and Sun (Ethan's testing right now). Start composing the "all clear" mail right now -- I think Ethan's "thumbs up" is coming shortly. On Aug 27, 2007, at 11:35 AM, Josh Hursey wrote: I say we are good to go since we are approaching Noon. We can work out the bugs as we go. What do you all think? -- Josh On Aug 27, 2007, at 10:50 AM, Jeff Squyres wrote: BTW, I think we're good to go. Ethan? On Aug 27, 2007, at 10:17 AM, Josh Hursey wrote: Fixed. On Aug 27, 2007, at 10:03 AM, Jeff Squyres wrote: Ok, it's looking good so far (I accidentally submitted one intel mpi result as a non "trial" -- oops). What's "Filter tracked failures" in the preferences window? The link on "tracked" goes to a 404. On Aug 27, 2007, at 9:48 AM, Jeff Squyres wrote: Let's do some simple submits and ensure that we can query for them properly. I'm guessing that everything will be fine, but let's be thorough... I'll kick off a simple round of get/install/get/build/run for trivial tests now. On Aug 27, 2007, at 9:28 AM, Josh Hursey wrote: Yeah I was unsure if I had the proper permissions to move the file. Apparently I do :) The upgrade is finished now. I just ran a simple MTT on the Odin machine at IU and it seemed to have gone through well. Should we announce to the larger list or did you all want to do some testing first? -- josh On Aug 27, 2007, at 7:48 AM, Jeff Squyres wrote: I think probably the easiest thing to do is just mv reporter.php and submit.php so that mtt clients and individuals browsing will not be able to access it. On Aug 26, 2007, at 10:58 PM, Josh Hursey wrote: Jeff and Ethan, How do I shutdown the MTT server? Do I just log into milliways as mtt, or can I disable it from my user? Cheers, -- Josh Begin forwarded message: From: Josh Hursey Date: August 24, 2007 1:37:18 PM EDT To: General user list for the MPI Testing Tool users@open- mpi.org> Subject: [MTT users] MTT Database and Reporter Upgrade **Action Required** Reply-To: General user list for the MPI Testing Tool Short Version: -- The MTT development group is rolling out newly optimized web frontend and backend database. As a result we will be taking down the MTT site at IU Monday, August 27 from 8 am to Noon US eastern time. During this time you will not be able to submit data to the MTT database. Therefore you need to disable any runs that will report during this time or your client will fail with unable to connect to server messages. This change does not affect the client configurations, so MTT users do *not* need to update their clients at this time. Longer Version: --- The MTT development team has been working diligently on server side optimizations over the past few months. This work involved major changes to the database schema, web reporter, and web submit components of the server. We want to roll out the new server side optimizations on Monday, Aug. 27. Given the extensive nature of the improvements the MTT server will need to be taken down for a few hours for this upgrade to take place. We are planning on taking down the MTT server at 8 am and we hope to have it back by Noon US Eastern time. MTT users that would normally submit results during this time range will need to disable their runs, or they will see server error messages during this outage. This upgrade does not require any client changes, so outside of the down time contributors need not change or upgrade their MTT installations. Below are a few rough performance numbers illustrating the difference between the old and new server versions as seen by the reporter. Summary report: 24 hours, all orgs 87 sec - old version 6 sec - new version Summary report: 24 hours, org = 'iu' 37 sec - old 4 sec - new Summary report: Past 3 days, all orgs 138 sec - old 9 sec - new Summary report: Past 3 days, org = 'iu' 49 sec - old 11 sec - new Summary report: Past 2 weeks, all orgs 863 sec - old 34 sec - new Summary report: Past 2 weeks, org = 'iu' 878 sec - old 12 sec - new Summary report: Past 1 month, all org 1395 sec - old 158 sec - new Summary report: Past 1 month, org = 'iu' 1069 sec - old 39 sec - new Summary report: (2007-06-18 - 2007-06-19), all org 484 sec - old 5 sec - new Summary report: (2007-06-18 - 2007-06-19), org = 'iu' 479 sec - old 2 sec - new ___ mtt-users mailing list mtt-us...@open-mpi.org http://www.open-mpi.org/mailman/listinfo.cgi/mtt-users ___ mtt-devel mailing list mtt-de...@open-mpi.org http://www.open-mpi.org/mailman/listinfo.cgi/mtt-devel -- Jeff Squyres Cisco Systems ___ mtt-devel mailing list mtt-de...@open-mpi.org http://www.open-mpi.org/mailman/listinfo.cgi/mtt-devel __
Re: [MTT devel] Thoughts on tagging...
With regards to a Gmail-style interface for labeling, got a comment/concern on the SQL. IIRC, when cherry-picking was implemented (for performance reports), we attempted to compose a WHERE clause that would effect the cherry-pick. This led to some wild WHERE clauses. E.g., consider the following example: +---++++---+ |# |A |B |C |D | +---++++---+ |1 |1 |2 |3 |4 | |2 |5 |6 |7 |8 | |3 |9 |10 |11 |12 | |4 |28 |10 |11 |12 | |5 |29 |10 |11 |12 | +---++++---+ If we want to cherry-pick lines 1, 2 and 3, then our WHERE clause will look like this: SELECT blah blah FROM blah,blah WHERE (A=1 AND B=2 AND C=3 AND D=4) OR (A=5 AND B=6 AND C=7 AND D=8) OR (A=9 AND B=10 AND C=11 AND D=12) We eventually chose do the filtering in PHP on row # since there does not seem to be a good way to filter by row # in SQL. The point being, a nasty WHERE clause *could* lead to a lng tag operation. Given the above, a good starting point might be to restrict tagging to the following: 1. Allow tagging only on entire reports 2. Allow tagging only on a single row at a time My $0.02. -Ethan On Fri, Aug/24/2007 03:07:54PM, Jeff Squyres wrote: > I volunteered to do this on the call today. Here's my thoughts on > tagging: > > 1. From the client, it would be nice to be able to specify a comma- > delimited list of tags at any phase. Tags would be inherited by > successive phases if not explicitly overridden. E.g., if you specify > a "foo" tag in an MPI get, it'll be used in all phases that use that > MPI get. > > Tags can be specified in one of three forms: > >+foo: means to *add* this tag to the existing/inherited set >-foo: means to *remove* this tag from the existing/inherited set >foo: if any tag does not have a +/- prefix, then the inherited set > is cleared, effectively making the current set of tags be only the > non-prefixed tags and +tags > > For example: > >[MPI Get: AAA] ># + and - have little meaning for MPI Get >tags = foo, bar, baz > >[Test Get: BBB] ># + and - have little meaning for Test Get >tags = yar, fweezle, bozzle > >[Test Build: CCC] ># Test build inherits tags from MPI Get and Test Get >tags = +fa-schizzle, -yar ># Resulting tag set: foo, bar, baz, fweezle, bozzle, fa-schizzle > >[Test build: DDD] ># Override everything >tags = yowza, gurple ># Resulting tag set: yowza, gurple > > 2. For the reporter, I think we only want authenticated users to be > able to create / manipulate tags. Authentication can be via SVN > username / password or the HTTPS submit username / password; I don't > have strong preferences. > > Anyone can query on tags, of course. > > 3. We should have easy "add these results to a tag" and "remove these > results from a tag" operations, similar to GMail/labels. I think the > rule should be that if you can show MPI details (i.e., not the > summary page), you can add/remove tags. Perhaps something as simple > as a text box with two buttons: Add tag, Remove tag. > > 3a. Example: you drill down to a set of test runs. You type in "jeff > results" in the text box and click the "add tag" button. This adds > the tag "jeff results" to all the result rows that are checked (it is > not an error if the "jeff results" tag already exists on some/all of > the result rows). > > 3b. Example: you drill down to a set of test runs. You type in "jeff > results" in the text box and click on the "remove tag" button. This > removes the tag "jeff results" from all the result rows that are > checked (it is not an error if the jeff results" tag is not on some/ > all of the result rows). > > 4. Per Gmail index label listing, it would be nice to see a list of > tags that exist on a given result row. It could be as simple as > adding another show/hide column for the tags on a given result row. > But it gets a little more complicated because one row many represent > many different results -- do we show the union of tags for all the > rollup rows? Maybe we can use different colors / attributes to > represent "this tag exists on *some* of the results in this row" vs. > "this tag exists on *all* of the results in this row"...? > > 4a. If the tags are listed as a column, they should also (of course) > be clickable so that if you click on them, you get the entire set of > results associated with that tag. > > 4b. For every tag on a rollup row, it would be good to be able to say > "apply this tag to every result in this rollup row" (i.e., this tag > had previously only applied to *some* of the results in this rollup > row). This could be displayed as a little "+" icon next to the tag > name, or somesuch. > > 4c. Similarly, for every tag, it would be good to have a "remove this > tag from every result in this row". This could be displayed as a >
Re: [MTT devel] Performance Graph Notes
On Mon, Aug/27/2007 10:33:26AM, Josh Hursey wrote: > Jeff asked me if I would talk to Rich and folks at IU on Friday about > the performance graphs. Below are my notes from that meeting: > > - It would be really useful to 'zoom' into sections of the graph. > Primarily restricting the x-axis (Message Size), but also having the > ability to restrict the y-axis (time) > > - Calling the y-axis 'latency' is a bit misleading, maybe 'time' > would be better. Minor issue. > > - Torsten mentioned that he was interested in seeing the other skampi > data that we are throwing away. Namely the time-per-rank. And if > available communicator size. Torsten should be able to click the Detail button from Performance view, and see everything that went to stdout in the test. Would that show time-per-rank? > > - Torsten mentioned that he wants to add some non-blocking collective > test that he is work on. I told him to contact Jeff on how to do this. > > - We need a well defined way to see what collective implementation > was used. Meaning that there are N AlltoAll collective > implementations in the 'tuned' component we need to know when looking > at the graph which one of the N we are looking at for Open MPI. For > other implementations we don't have so much control. > > - It is difficult to search in the reporter for queries like: > -- > * Open MPI run with only tcp,sm,self > How about something like this? http://www.open-mpi.org/mtt/reporter.php?do_redir=288 -Ethan > * Intel MPI (which is only tcp I believe) > * MPICH2 with tcp results from running the skampi Bcast benchmark. > -- > The reporter is designed to track a single MPI well for regression > tracking. However when we need to compare multiple MPIs and each may > need to be selected with a different type of query it is impossible/ > hard to do. > One solution I proposed was using the 'tagging' idea, but there might > be some alternative UI features that we can develop to better support > these types of queries. Tim P seemed interested/had some ideas on how > to do this. > > - They really liked the ability to look at the HTML version of the > raw data. They seemed frustrated that the popup window is reused when > looking at multiple HTML versions of the raw data. They wanted this > to be a static window that they could keep open so they could look at > multiple variants of this data in small screens. > > That was about it. They thought it was good over all, but the above > were suggestions on ways to make the representation more useful. > > -- Josh > > ___ > mtt-devel mailing list > mtt-de...@open-mpi.org > http://www.open-mpi.org/mailman/listinfo.cgi/mtt-devel
Re: [MTT devel] MTT Database and Reporter Upgrade **Action Required**
On Mon, Aug/27/2007 11:41:26AM, Jeff Squyres wrote: > I see a bunch of submits in the db -- including some from > IBM and Sun (Ethan's testing right now). > > Start composing the "all clear" mail right now -- I think > Ethan's "thumbs up" is coming shortly. > My "Trivial" test runs just submitted fine. Looks good! -Ethan > > On Aug 27, 2007, at 11:35 AM, Josh Hursey wrote: > > > I say we are good to go since we are approaching Noon. We can work > > out the bugs as we go. > > > > What do you all think? > > -- Josh > > > > On Aug 27, 2007, at 10:50 AM, Jeff Squyres wrote: > > > >> BTW, I think we're good to go. Ethan? > >> > >> > >> On Aug 27, 2007, at 10:17 AM, Josh Hursey wrote: > >> > >>> Fixed. > >>> > >>> On Aug 27, 2007, at 10:03 AM, Jeff Squyres wrote: > >>> > Ok, it's looking good so far (I accidentally submitted one intel > mpi > result as a non "trial" -- oops). > > What's "Filter tracked failures" in the preferences window? > > The link on "tracked" goes to a 404. > > > > On Aug 27, 2007, at 9:48 AM, Jeff Squyres wrote: > > > Let's do some simple submits and ensure that we can query for them > > properly. I'm guessing that everything will be fine, but let's be > > thorough... > > > > I'll kick off a simple round of get/install/get/build/run for > > trivial > > tests now. > > > > On Aug 27, 2007, at 9:28 AM, Josh Hursey wrote: > > > >> Yeah I was unsure if I had the proper permissions to move the > >> file. > >> Apparently I do :) > >> > >> The upgrade is finished now. I just ran a simple MTT on the Odin > >> machine at IU and it seemed to have gone through well. Should we > >> announce to the larger list or did you all want to do some > >> testing > >> first? > >> > >> -- josh > >> > >> On Aug 27, 2007, at 7:48 AM, Jeff Squyres wrote: > >> > >>> I think probably the easiest thing to do is just mv reporter.php > >>> and > >>> submit.php so that mtt clients and individuals browsing will > >>> not be > >>> able to access it. > >>> > >>> > >>> On Aug 26, 2007, at 10:58 PM, Josh Hursey wrote: > >>> > Jeff and Ethan, > > How do I shutdown the MTT server? Do I just log into > milliways as > mtt, or can I disable it from my user? > > Cheers, > -- Josh > > Begin forwarded message: > > > From: Josh Hursey > > Date: August 24, 2007 1:37:18 PM EDT > > To: General user list for the MPI Testing Tool > users@open- > > mpi.org> > > Subject: [MTT users] MTT Database and Reporter Upgrade > > **Action > > Required** > > Reply-To: General user list for the MPI Testing Tool > us...@open-mpi.org> > > > > Short Version: > > -- > > The MTT development group is rolling out newly optimized web > > frontend > > and backend database. As a result we will be taking down the > > MTT > > site > > at IU Monday, August 27 from 8 am to Noon US eastern time. > > > > During this time you will not be able to submit data to the > > MTT > > database. Therefore you need to disable any runs that will > > report > > during this time or your client will fail with unable to > > connect to > > server messages. > > > > This change does not affect the client configurations, so MTT > > users > > do *not* need to update their clients at this time. > > > > > > Longer Version: > > --- > > The MTT development team has been working diligently on server > > side > > optimizations over the past few months. This work involved > > major > > changes to the database schema, web reporter, and web submit > > components of the server. > > > > We want to roll out the new server side optimizations on > > Monday, > > Aug. > > 27. Given the extensive nature of the improvements the MTT > > server > > will need to be taken down for a few hours for this upgrade to > > take > > place. We are planning on taking down the MTT server at 8 am > > and > > we hope to have it back by Noon US Eastern time. > > > > MTT users that would normally submit results during this time > > range > > will need to disable their runs, or they will see server error > > messages during this outage. > > > > This upgrade does not require any client changes, so > > outside of > > the > > down time contributors need not change or upgrade their MTT > > installations. >
[MTT devel] jms-new-parser / submit.php error
I'm running into the below error running with the jms-new-parser branch
(see attached MTTDatabase error file).
*** WARNING: MTTDatabase server notice: mpi_install_section_name is not in mtt
database.
MTTDatabase server notice: number_of_results is not in mtt database.
MTTDatabase server notice: phase is not in mtt database.
MTTDatabase server notice: fields is not in mtt database.
MTTDatabase server notice: mpi_get_section_name is not in mtt database.
MTT submission for test build
MTTDatabase server error:
SQL QUERY: SELECT mpi_install_id
FROM mpi_install NATURAL JOIN
mpi_get NATURAL JOIN
compilerNATURAL JOIN
compute_cluster NATURAL JOIN
submit
WHERE
mpi_version = DEFAULT AND
mpi_name= 'clustertools-7-iso-sdn-0907' AND
compiler_version = '5.9 2007/05/03;' AND
compiler_name= 'sun' AND
hostname = 'burl-ct-v440-2' AND
mtt_client_version = '2.1devel' AND
local_username = 'emallove' AND
platform_name= 'burl-ct-v440-2'
ORDER BY mpi_install_id DESC limit 1
SQL ERROR: ERROR: syntax error at or near "DEFAULT"
LINE 8: mpi_version = DEFAULT AND
^
SQL ERROR:
MTTDatabase server notice: Invalid mpi_install_id (9790) given. Guessing
that it should be -1
MTTDatabase server error: ERROR: Unable to find a mpi_install to associate
with this test_build.
MTTDatabase abort: (Tried to send HTTP error) 400
MTTDatabase abort:
No mpi_install associated with this test_build
MTTDatabase got response: MTTDatabase server notice: mpi_install_section_name
is not in mtt database.
MTTDatabase server notice: number_of_results is not in mtt database.
MTTDatabase server notice: phase is not in mtt database.
MTTDatabase server notice: fields is not in mtt database.
MTTDatabase server notice: mpi_get_section_name is not in mtt database.
MTT submission for test build
MTTDatabase server error:
SQL QUERY: SELECT mpi_install_id
FROM mpi_install NATURAL JOIN
mpi_get NATURAL JOIN
compilerNATURAL JOIN
compute_cluster NATURAL JOIN
submit
WHERE
mpi_version = DEFAULT AND
mpi_name= 'clustertools-7-iso-sdn-0907' AND
compiler_version = '5.9 2007/05/03;' AND
compiler_name= 'sun' AND
hostname = 'burl-ct-v440-2' AND
mtt_client_version = '2.1devel' AND
local_username = 'emallove' AND
platform_name= 'burl-ct-v440-2'
ORDER BY mpi_install_id DESC limit 1
SQL ERROR: ERROR: syntax error at or near "DEFAULT"
LINE 8: mpi_version = DEFAULT AND
^
SQL ERROR:
MTTDatabase server notice: Invalid mpi_install_id (9790) given. Guessing
that it should be -1
MTTDatabase server error: ERROR: Unable to find a mpi_install to associate
with this test_build.
MTTDatabase abort: (Tried to send HTTP error) 400
MTTDatabase abort:
No mpi_install associated with this test_build
*** WARNING: MTTDatabase did not get a serial; phases will be isolated from
each other in the reports
MTTDatabase submit complete
Writing to MTTDatabase debug file:
/home/em162155/mtt-utils/logs/debug/mttdatabase.burl-ct-v440-2.20070827.153417.1.1188243271-error.txt
Debug MTTDatabase file write complete
>> Reported to MTTDatabase: 1 successful submit, 0 failed submits (total of 1
>> result)
# *** WARNING:
# 2 MTTDatabase server errors
# The data that failed to submit is in
/home/em162155/mtt-utils/logs/debug/mttdatabase.burl-ct-v440-2.20070827.153417.*.txt.
# See the above output for more info.
$VAR1 = {
'resource_manager_14' => 'unknown',
'resource_manager_8' => 'unknown',
'environment_20' => '',
'parameters_18' => '',
'test_result_12' => 1,
'result_message_13' => 'Passed',
'mpi_install_section_name_1' => 'clustertools-7-iso-sdn-0907',
'result_stderr_23' => undef,
'test_result_9' => 1,
'result_message_24' => 'Passed',
'result_stdout_23' => '[burl-ct-v440-2:10577] Error in mx_init (error
No MX device entry in /dev.)
[burl-ct-v440-2:10577] mca_btl_mx_component_init: mx_get_info(MX_NIC_COUNT)
failed with status=4
[burl-ct-v440-2:10579] Error in mx_init (error No MX device entry in /dev.)
[burl-ct-v440-2:10579] mca_btl_mx_component_init: mx_get_info(MX_NIC_COUNT)
failed with status=4
[burl-ct-v440-2:10583] Error in mx_init (error No MX device entry in /dev.)
[burl-ct-v440-2:10583] mca_btl_mx_component_init: mx_get_info(MX_NIC_COUNT)
failed with status=4
[burl-ct-v440-2:10581] Error in mx_init (error N
Re: [MTT devel] jms-new-parser / submit.php error
Well that's fun -- why are there no mpi_install values in the .txt file? I assume you had successful MPI installs before this? On Aug 27, 2007, at 4:03 PM, Ethan Mallove wrote: I'm running into the below error running with the jms-new-parser branch (see attached MTTDatabase error file). *** WARNING: MTTDatabase server notice: mpi_install_section_name is not in mtt database. MTTDatabase server notice: number_of_results is not in mtt database. MTTDatabase server notice: phase is not in mtt database. MTTDatabase server notice: fields is not in mtt database. MTTDatabase server notice: mpi_get_section_name is not in mtt database. MTT submission for test build MTTDatabase server error: SQL QUERY: SELECT mpi_install_id FROM mpi_install NATURAL JOIN mpi_get NATURAL JOIN compilerNATURAL JOIN compute_cluster NATURAL JOIN submit WHERE mpi_version = DEFAULT AND mpi_name= 'clustertools-7-iso-sdn-0907' AND compiler_version = '5.9 2007/05/03;' AND compiler_name= 'sun' AND hostname = 'burl-ct-v440-2' AND mtt_client_version = '2.1devel' AND local_username = 'emallove' AND platform_name= 'burl-ct-v440-2' ORDER BY mpi_install_id DESC limit 1 SQL ERROR: ERROR: syntax error at or near "DEFAULT" LINE 8: mpi_version = DEFAULT AND ^ SQL ERROR: MTTDatabase server notice: Invalid mpi_install_id (9790) given. Guessing that it should be -1 MTTDatabase server error: ERROR: Unable to find a mpi_install to associate with this test_build. MTTDatabase abort: (Tried to send HTTP error) 400 MTTDatabase abort: No mpi_install associated with this test_build MTTDatabase got response: MTTDatabase server notice: mpi_install_section_name is not in mtt database. MTTDatabase server notice: number_of_results is not in mtt database. MTTDatabase server notice: phase is not in mtt database. MTTDatabase server notice: fields is not in mtt database. MTTDatabase server notice: mpi_get_section_name is not in mtt database. MTT submission for test build MTTDatabase server error: SQL QUERY: SELECT mpi_install_id FROM mpi_install NATURAL JOIN mpi_get NATURAL JOIN compilerNATURAL JOIN compute_cluster NATURAL JOIN submit WHERE mpi_version = DEFAULT AND mpi_name= 'clustertools-7-iso-sdn-0907' AND compiler_version = '5.9 2007/05/03;' AND compiler_name= 'sun' AND hostname = 'burl-ct-v440-2' AND mtt_client_version = '2.1devel' AND local_username = 'emallove' AND platform_name= 'burl-ct-v440-2' ORDER BY mpi_install_id DESC limit 1 SQL ERROR: ERROR: syntax error at or near "DEFAULT" LINE 8: mpi_version = DEFAULT AND ^ SQL ERROR: MTTDatabase server notice: Invalid mpi_install_id (9790) given. Guessing that it should be -1 MTTDatabase server error: ERROR: Unable to find a mpi_install to associate with this test_build. MTTDatabase abort: (Tried to send HTTP error) 400 MTTDatabase abort: No mpi_install associated with this test_build *** WARNING: MTTDatabase did not get a serial; phases will be isolated from each other in the reports MTTDatabase submit complete Writing to MTTDatabase debug file: /home/em162155/mtt-utils/logs/ debug/mttdatabase.burl-ct-v440-2.20070827.153417.1.1188243271- error.txt Debug MTTDatabase file write complete Reported to MTTDatabase: 1 successful submit, 0 failed submits (total of 1 result) ## ## # *** WARNING: # 2 MTTDatabase server errors # The data that failed to submit is in /home/em162155/mtt-utils/ logs/debug/mttdatabase.burl-ct-v440-2.20070827.153417.*.txt. # See the above output for more info. ## ## ___ mtt-devel mailing list mtt-de...@open-mpi.org http://www.open-mpi.org/mailman/listinfo.cgi/mtt-devel -- Jeff Squyres Cisco Systems
Re: [MTT devel] jms-new-parser / submit.php error
This is odd. The test_build submission code is trying to guess the mpi_install_id, and since the "mpi_version=''" in the .txt file then this resolves as DEFAULT on the server which is no good. Was it that the mpi_version was never filled in by the client or was it improperly reported? -- Josh On Aug 27, 2007, at 4:07 PM, Jeff Squyres wrote: Well that's fun -- why are there no mpi_install values in the .txt file? I assume you had successful MPI installs before this? On Aug 27, 2007, at 4:03 PM, Ethan Mallove wrote: I'm running into the below error running with the jms-new-parser branch (see attached MTTDatabase error file). *** WARNING: MTTDatabase server notice: mpi_install_section_name is not in mtt database. MTTDatabase server notice: number_of_results is not in mtt database. MTTDatabase server notice: phase is not in mtt database. MTTDatabase server notice: fields is not in mtt database. MTTDatabase server notice: mpi_get_section_name is not in mtt database. MTT submission for test build MTTDatabase server error: SQL QUERY: SELECT mpi_install_id FROM mpi_install NATURAL JOIN mpi_get NATURAL JOIN compilerNATURAL JOIN compute_cluster NATURAL JOIN submit WHERE mpi_version = DEFAULT AND mpi_name= 'clustertools-7-iso-sdn-0907' AND compiler_version = '5.9 2007/05/03;' AND compiler_name= 'sun' AND hostname = 'burl-ct-v440-2' AND mtt_client_version = '2.1devel' AND local_username = 'emallove' AND platform_name= 'burl-ct-v440-2' ORDER BY mpi_install_id DESC limit 1 SQL ERROR: ERROR: syntax error at or near "DEFAULT" LINE 8: mpi_version = DEFAULT AND ^ SQL ERROR: MTTDatabase server notice: Invalid mpi_install_id (9790) given. Guessing that it should be -1 MTTDatabase server error: ERROR: Unable to find a mpi_install to associate with this test_build. MTTDatabase abort: (Tried to send HTTP error) 400 MTTDatabase abort: No mpi_install associated with this test_build MTTDatabase got response: MTTDatabase server notice: mpi_install_section_name is not in mtt database. MTTDatabase server notice: number_of_results is not in mtt database. MTTDatabase server notice: phase is not in mtt database. MTTDatabase server notice: fields is not in mtt database. MTTDatabase server notice: mpi_get_section_name is not in mtt database. MTT submission for test build MTTDatabase server error: SQL QUERY: SELECT mpi_install_id FROM mpi_install NATURAL JOIN mpi_get NATURAL JOIN compilerNATURAL JOIN compute_cluster NATURAL JOIN submit WHERE mpi_version = DEFAULT AND mpi_name= 'clustertools-7-iso-sdn-0907' AND compiler_version = '5.9 2007/05/03;' AND compiler_name= 'sun' AND hostname = 'burl-ct-v440-2' AND mtt_client_version = '2.1devel' AND local_username = 'emallove' AND platform_name= 'burl-ct-v440-2' ORDER BY mpi_install_id DESC limit 1 SQL ERROR: ERROR: syntax error at or near "DEFAULT" LINE 8: mpi_version = DEFAULT AND ^ SQL ERROR: MTTDatabase server notice: Invalid mpi_install_id (9790) given. Guessing that it should be -1 MTTDatabase server error: ERROR: Unable to find a mpi_install to associate with this test_build. MTTDatabase abort: (Tried to send HTTP error) 400 MTTDatabase abort: No mpi_install associated with this test_build *** WARNING: MTTDatabase did not get a serial; phases will be isolated from each other in the reports MTTDatabase submit complete Writing to MTTDatabase debug file: /home/em162155/mtt-utils/logs/ debug/mttdatabase.burl-ct-v440-2.20070827.153417.1.1188243271- error.txt Debug MTTDatabase file write complete Reported to MTTDatabase: 1 successful submit, 0 failed submits (total of 1 result) # # ## # *** WARNING: # 2 MTTDatabase server errors # The data that failed to submit is in /home/em162155/mtt-utils/ logs/debug/mttdatabase.burl-ct-v440-2.20070827.153417.*.txt. # See the above output for more info. # # ## ___ mtt-devel mailing list mtt-de...@open-mpi.org http://www.open-mpi.org/mailman/listinfo.cgi/mtt-devel -- Jeff Squyres Cisco Systems ___ mtt-devel mailing list mtt-de...@open-mpi.org http://www.open-mpi.org/mailman/listinfo.cgi/mtt-devel
Re: [MTT devel] jms-new-parser / submit.php error
Blah -- I *meant* to say mpi_install_id, not mpi_install (i.e., there's no mpi_install_id values). *And* the mpi_version string is "", as Josh noted. What happened with your prior MPI install's? On Aug 27, 2007, at 5:00 PM, Josh Hursey wrote: This is odd. The test_build submission code is trying to guess the mpi_install_id, and since the "mpi_version=''" in the .txt file then this resolves as DEFAULT on the server which is no good. Was it that the mpi_version was never filled in by the client or was it improperly reported? -- Josh On Aug 27, 2007, at 4:07 PM, Jeff Squyres wrote: Well that's fun -- why are there no mpi_install values in the .txt file? I assume you had successful MPI installs before this? On Aug 27, 2007, at 4:03 PM, Ethan Mallove wrote: I'm running into the below error running with the jms-new-parser branch (see attached MTTDatabase error file). *** WARNING: MTTDatabase server notice: mpi_install_section_name is not in mtt database. MTTDatabase server notice: number_of_results is not in mtt database. MTTDatabase server notice: phase is not in mtt database. MTTDatabase server notice: fields is not in mtt database. MTTDatabase server notice: mpi_get_section_name is not in mtt database. MTT submission for test build MTTDatabase server error: SQL QUERY: SELECT mpi_install_id FROM mpi_install NATURAL JOIN mpi_get NATURAL JOIN compilerNATURAL JOIN compute_cluster NATURAL JOIN submit WHERE mpi_version = DEFAULT AND mpi_name= 'clustertools-7-iso-sdn-0907' AND compiler_version = '5.9 2007/05/03;' AND compiler_name= 'sun' AND hostname = 'burl-ct-v440-2' AND mtt_client_version = '2.1devel' AND local_username = 'emallove' AND platform_name= 'burl-ct-v440-2' ORDER BY mpi_install_id DESC limit 1 SQL ERROR: ERROR: syntax error at or near "DEFAULT" LINE 8: mpi_version = DEFAULT AND ^ SQL ERROR: MTTDatabase server notice: Invalid mpi_install_id (9790) given. Guessing that it should be -1 MTTDatabase server error: ERROR: Unable to find a mpi_install to associate with this test_build. MTTDatabase abort: (Tried to send HTTP error) 400 MTTDatabase abort: No mpi_install associated with this test_build MTTDatabase got response: MTTDatabase server notice: mpi_install_section_name is not in mtt database. MTTDatabase server notice: number_of_results is not in mtt database. MTTDatabase server notice: phase is not in mtt database. MTTDatabase server notice: fields is not in mtt database. MTTDatabase server notice: mpi_get_section_name is not in mtt database. MTT submission for test build MTTDatabase server error: SQL QUERY: SELECT mpi_install_id FROM mpi_install NATURAL JOIN mpi_get NATURAL JOIN compilerNATURAL JOIN compute_cluster NATURAL JOIN submit WHERE mpi_version = DEFAULT AND mpi_name= 'clustertools-7-iso-sdn-0907' AND compiler_version = '5.9 2007/05/03;' AND compiler_name= 'sun' AND hostname = 'burl-ct-v440-2' AND mtt_client_version = '2.1devel' AND local_username = 'emallove' AND platform_name= 'burl-ct-v440-2' ORDER BY mpi_install_id DESC limit 1 SQL ERROR: ERROR: syntax error at or near "DEFAULT" LINE 8: mpi_version = DEFAULT AND ^ SQL ERROR: MTTDatabase server notice: Invalid mpi_install_id (9790) given. Guessing that it should be -1 MTTDatabase server error: ERROR: Unable to find a mpi_install to associate with this test_build. MTTDatabase abort: (Tried to send HTTP error) 400 MTTDatabase abort: No mpi_install associated with this test_build *** WARNING: MTTDatabase did not get a serial; phases will be isolated from each other in the reports MTTDatabase submit complete Writing to MTTDatabase debug file: /home/em162155/mtt-utils/logs/ debug/mttdatabase.burl-ct-v440-2.20070827.153417.1.1188243271- error.txt Debug MTTDatabase file write complete Reported to MTTDatabase: 1 successful submit, 0 failed submits (total of 1 result) # # ## # *** WARNING: # 2 MTTDatabase server errors # The data that failed to submit is in /home/em162155/mtt-utils/ logs/debug/mttdatabase.burl-ct-v440-2.20070827.153417.*.txt. # See the above output for more info. # # ## ___ mtt-devel mailing list mtt-de...@open-mpi.org http://www.open-mpi.org/mailman/listinfo.cgi/mtt-devel -- Jeff Squyres Cisco Systems ___ mtt-devel mailing list mtt-de...@open-mpi.org http://www.o
Re: [MTT devel] jms-new-parser / submit.php error
On Mon, Aug/27/2007 04:07:57PM, Jeff Squyres wrote: > Well that's fun -- why are there no mpi_install values in the .txt file? > Because Functions/MPI/OMPI::get_version() does not know what my $bindir is. Should an &installdir() funclet be created to get around this? -Ethan > I assume you had successful MPI installs before this? > > > On Aug 27, 2007, at 4:03 PM, Ethan Mallove wrote: > > > I'm running into the below error running with the jms-new-parser > > branch > > (see attached MTTDatabase error file). > > > > *** WARNING: MTTDatabase server notice: mpi_install_section_name is > > not in mtt database. > > MTTDatabase server notice: number_of_results is not in mtt > > database. > > MTTDatabase server notice: phase is not in mtt database. > > MTTDatabase server notice: fields is not in mtt database. > > MTTDatabase server notice: mpi_get_section_name is not in mtt > > database. > > > > MTT submission for test build > > MTTDatabase server error: > > SQL QUERY: SELECT mpi_install_id > > FROM mpi_install NATURAL JOIN > > mpi_get NATURAL JOIN > > compilerNATURAL JOIN > > compute_cluster NATURAL JOIN > > submit > > WHERE > > mpi_version = DEFAULT AND > > mpi_name= 'clustertools-7-iso-sdn-0907' AND > > compiler_version = '5.9 2007/05/03;' AND > > compiler_name= 'sun' AND > > hostname = 'burl-ct-v440-2' AND > > mtt_client_version = '2.1devel' AND > > local_username = 'emallove' AND > > platform_name= 'burl-ct-v440-2' > > ORDER BY mpi_install_id DESC limit 1 > > SQL ERROR: ERROR: syntax error at or near "DEFAULT" > > LINE 8: mpi_version = DEFAULT AND > >^ > > SQL ERROR: > > MTTDatabase server notice: Invalid mpi_install_id (9790) given. > > Guessing that it should be -1 > > MTTDatabase server error: ERROR: Unable to find a mpi_install > > to associate with this test_build. > > > > MTTDatabase abort: (Tried to send HTTP error) 400 > > MTTDatabase abort: > > No mpi_install associated with this test_build > > MTTDatabase got response: MTTDatabase server notice: > > mpi_install_section_name is not in mtt database. > >MTTDatabase server notice: number_of_results is not in mtt > > database. > >MTTDatabase server notice: phase is not in mtt database. > >MTTDatabase server notice: fields is not in mtt database. > >MTTDatabase server notice: mpi_get_section_name is not in mtt > > database. > > > >MTT submission for test build > >MTTDatabase server error: > >SQL QUERY: SELECT mpi_install_id > >FROM mpi_install NATURAL JOIN > > mpi_get NATURAL JOIN > > compilerNATURAL JOIN > > compute_cluster NATURAL JOIN > > submit > >WHERE > >mpi_version = DEFAULT AND > >mpi_name= 'clustertools-7-iso-sdn-0907' AND > >compiler_version = '5.9 2007/05/03;' AND > >compiler_name= 'sun' AND > >hostname = 'burl-ct-v440-2' AND > >mtt_client_version = '2.1devel' AND > >local_username = 'emallove' AND > >platform_name= 'burl-ct-v440-2' > >ORDER BY mpi_install_id DESC limit 1 > >SQL ERROR: ERROR: syntax error at or near "DEFAULT" > >LINE 8: mpi_version = DEFAULT AND > > ^ > >SQL ERROR: > >MTTDatabase server notice: Invalid mpi_install_id (9790) given. > > Guessing that it should be -1 > >MTTDatabase server error: ERROR: Unable to find a mpi_install to > > associate with this test_build. > > > >MTTDatabase abort: (Tried to send HTTP error) 400 > >MTTDatabase abort: > >No mpi_install associated with this test_build > > *** WARNING: MTTDatabase did not get a serial; phases will be > > isolated from each other in the reports > > MTTDatabase submit complete > > Writing to MTTDatabase debug file: /home/em162155/mtt-utils/logs/ > > debug/mttdatabase.burl-ct-v440-2.20070827.153417.1.1188243271- > > error.txt > > Debug MTTDatabase file write complete > >>> Reported to MTTDatabase: 1 successful submit, 0 failed submits > >>> (total of 1 result) > > > > ## > > ## > > # *** WARNING: > > # 2 MTTDatabase server errors > > # The data that failed to submit is in /home/em162155/mtt-utils/ > > logs/debug/mttdatabase.burl-ct-v440-2.20070827.153417.*.txt. > > # See the above output for more info. > > ## > > ## > > > > ___ > > mtt-devel mailing list > > mtt-de...@open-mpi.org > > http://www.open-mpi.org/mailman/listinfo.cgi/mtt-devel > > > -- > Jeff Squyres > Cisco Systems > > ___ > mtt-devel mailing list > mtt-de...@open-mpi.org > http://www.open-mpi.org/mailman/listinfo.cgi/mtt-d
Re: [MTT devel] Performance Graph Notes
On Aug 27, 2007, at 12:06 PM, Ethan Mallove wrote: - It would be really useful to 'zoom' into sections of the graph. Primarily restricting the x-axis (Message Size), but also having the ability to restrict the y-axis (time) K. https://svn.open-mpi.org/trac/mtt/ticket/153 - Calling the y-axis 'latency' is a bit misleading, maybe 'time' would be better. Minor issue. Easy to fix (and we should). https://svn.open-mpi.org/trac/mtt/ticket/285 - Torsten mentioned that he was interested in seeing the other skampi data that we are throwing away. Namely the time-per-rank. And if available communicator size. Torsten should be able to click the Detail button from Performance view, and see everything that went to stdout in the test. Would that show time-per-rank? Not ATM; the skampi build is currently configured to not show that info. Darn that Torsten... If we want, we can resume the discussion of how to save that info (since Jelena told us "no", I honestly dumped all that info from my brain...). - Torsten mentioned that he wants to add some non-blocking collective test that he is work on. I told him to contact Jeff on how to do this. Shouldn't be hard. I'll wait for him to contact me. - We need a well defined way to see what collective implementation was used. Meaning that there are N AlltoAll collective implementations in the 'tuned' component we need to know when looking at the graph which one of the N we are looking at for Open MPI. For other implementations we don't have so much control. I don't know if MTT can. In order for MTT to do this, OMPI needs to export that data somehow. - It is difficult to search in the reporter for queries like: -- * Open MPI run with only tcp,sm,self How about something like this? http://www.open-mpi.org/mtt/reporter.php?do_redir=288 I did some skampi runs to see verbs results across 2 MPIs (Intel MPI uses udapl, not tcp). I don't really think that this is hard: - network: verbs (or TCP in Josh's case) - test suite: skampi - command: bcast (granted, per #281, you have to fill in "bcast" on the "command" field on the advanced window, not the normal window) It should show all the MPI's. You probably want to limit it down to a specific platform, though, in order to get apples-to-apples comparisons. http://www.open-mpi.org/mtt/reporter.php?do_redir=290 * Intel MPI (which is only tcp I believe) * MPICH2 with tcp results from running the skampi Bcast benchmark. -- The reporter is designed to track a single MPI well for regression tracking. However when we need to compare multiple MPIs and each may need to be selected with a different type of query it is impossible/ hard to do. I don't see why this is hard...? I disagree with the statement "Reporter is design to track a single MPI well..." See the permalink above. One solution I proposed was using the 'tagging' idea, but there might be some alternative UI features that we can develop to better support these types of queries. Tim P seemed interested/had some ideas on how to do this. - They really liked the ability to look at the HTML version of the raw data. They seemed frustrated that the popup window is reused when looking at multiple HTML versions of the raw data. They wanted this to be a static window that they could keep open so they could look at multiple variants of this data in small screens. IIRC, that was some javascript trick that Ethan did in order to download everything once. It could probably be changed if someone really wanted to (e.g., the CSV doesn't display this way). Ethan, can you explain further? - The performance graphs are sometimes placed side-by-side instead of stacked on top of one another. This shinks the x-axis, and is undesirable. They would prefer that the graphs be always stacked on top of one another. That shouldn't be too hard, right Ethan? - They lamented the lack of the cherry picking feature since it is known to be broken in the new reporter. To be fixed... - They noticed that sometimes there is 'wasted space' in the graphs in both the x and y axis. They want the graph to be pushed to the edges of the graph so they can see the most detail in the results. This might be a function of our PHP graphing package. We seem to have jpgraph 1.20.5; the most recent seems to be 1.21b. I doubt this issue has been fixed, but we might check / ask...? -- Jeff Squyres Cisco Systems
Re: [MTT devel] jms-new-parser / submit.php error
On Aug 27, 2007, at 5:32 PM, Ethan Mallove wrote: Well that's fun -- why are there no mpi_install values in the .txt file? Because Functions/MPI/OMPI::get_version() does not know what my $bindir is. Should an &installdir() funclet be created to get around this? There's a chicken-n-egg problem here that I didn't solve and therefore ended up hard-coding for the pre-installed MPIs (HP, Intel, ...). The MPI Get phase is the one responsible for getting the MPI version. But AlreadyInstalled applies to a bunch of different MPIs -- each one has a different way of obtaining the version number (e.g., in OMPI, we call ompi_info). So it makes sense to have a funclet to get the version number for each different MPI (which I did). But the problem is that you need to know the $bindir in order to call the MPI's utility to get the version number. You *could* do something like: alreadyinstalled_version = &MPI::OMPI::get_version(&mpi_install_bindir ()) But there's two problems with this: 1. Then you have to ensure that &mpi_install_bindir() is valid from anywhere, meaning that we need to set some global variable that corresponds to the MPI install in use throughout the code base (e.g., even if you call it from within test get, test build, test run, ...). This is a PITA, but it's solvable; it's just annoying/ menial work to go track down everywhere in the code that needs to have this variable set. 2. We're trying to use an attribute from the MPI install phase (the bindir) in the MPI get phase (to get the version). This is a huge break in abstraction. All these funclets take a single param (the $bindir), but we won't know that until the MPI Install phase. How can we pass it during the MPI Get phase? I hadn't figured out how to do that without megga-ick/abstration breaks, so I gave up and hard- coded for HP/Intel MPI, etc. - So were you using AlreadyInstalled for OMPI to report back to the DB? Because I guess I thought we all understood that this was currently broken (per prior discussions on the phone). -Ethan I assume you had successful MPI installs before this? On Aug 27, 2007, at 4:03 PM, Ethan Mallove wrote: I'm running into the below error running with the jms-new-parser branch (see attached MTTDatabase error file). *** WARNING: MTTDatabase server notice: mpi_install_section_name is not in mtt database. MTTDatabase server notice: number_of_results is not in mtt database. MTTDatabase server notice: phase is not in mtt database. MTTDatabase server notice: fields is not in mtt database. MTTDatabase server notice: mpi_get_section_name is not in mtt database. MTT submission for test build MTTDatabase server error: SQL QUERY: SELECT mpi_install_id FROM mpi_install NATURAL JOIN mpi_get NATURAL JOIN compilerNATURAL JOIN compute_cluster NATURAL JOIN submit WHERE mpi_version = DEFAULT AND mpi_name= 'clustertools-7-iso-sdn-0907' AND compiler_version = '5.9 2007/05/03;' AND compiler_name= 'sun' AND hostname = 'burl-ct-v440-2' AND mtt_client_version = '2.1devel' AND local_username = 'emallove' AND platform_name= 'burl-ct-v440-2' ORDER BY mpi_install_id DESC limit 1 SQL ERROR: ERROR: syntax error at or near "DEFAULT" LINE 8: mpi_version = DEFAULT AND ^ SQL ERROR: MTTDatabase server notice: Invalid mpi_install_id (9790) given. Guessing that it should be -1 MTTDatabase server error: ERROR: Unable to find a mpi_install to associate with this test_build. MTTDatabase abort: (Tried to send HTTP error) 400 MTTDatabase abort: No mpi_install associated with this test_build MTTDatabase got response: MTTDatabase server notice: mpi_install_section_name is not in mtt database. MTTDatabase server notice: number_of_results is not in mtt database. MTTDatabase server notice: phase is not in mtt database. MTTDatabase server notice: fields is not in mtt database. MTTDatabase server notice: mpi_get_section_name is not in mtt database. MTT submission for test build MTTDatabase server error: SQL QUERY: SELECT mpi_install_id FROM mpi_install NATURAL JOIN mpi_get NATURAL JOIN compilerNATURAL JOIN compute_cluster NATURAL JOIN submit WHERE mpi_version = DEFAULT AND mpi_name= 'clustertools-7-iso-sdn-0907' AND compiler_version = '5.9 2007/05/03;' AND compiler_name= 'sun' AND hostname = 'burl-ct-v440-2' AND mtt_client_version = '2.1devel' AND local_username = 'emallove' AND platform_name= 'burl-ct-v440-2' ORDER BY mpi_install_id DESC limit 1 SQL ERROR: ERROR: syntax error at or near "DEFAULT" LINE 8: mpi_version =
mtt-devel@lists.open-mpi.org
Ethan --
You said to me in IM:
"i'm getting stuck trying to use MTT::Functions::find. it's returning
EVERY file under the directory i give it."
Can you cite a specific example? Is this on the jms-new-parser branch?
Keep in mind that you need to supply a *perl* regexp (not a shell
regexp). For example:
argv = -i &find("coll_.+.ski", "input_files")
--
Jeff Squyres
Cisco Systems
