Re: Qpid source code reorg update
Another update. CI is now in decent, if not perfect, shape. I've added lots of suppressions for apparently real memory leaks. I'll send a separate email regarding those. Now that the vote on the migration to git has passed, I've raised two INFRA jiras to start the migration process: https://issues.apache.org/jira/browse/INFRA-11794 https://issues.apache.org/jira/browse/INFRA-11795 On Tue, Apr 26, 2016 at 7:34 AM, Justin Ross wrote: > Hi, everyone. I'm now in the midst of getting the various CI environments > working, adding some temporary valgrind suppressions and fixing > incompatibilities with various versions of the buildchain tools. > > After CI is in suitably good shape, I would like to start the process to > migrate qpid-cpp and qpid-python to git. That will require a vote, which I > will kick off shortly. > > On Thu, Apr 21, 2016 at 7:06 AM, Justin Ross > wrote: > >> I've committed the major pieces of this, and I'm now doing more testing >> and improving the documentation. >> >> These changes are likely going to affect CI and test configurations. In >> particular, the cpp tree now depends on an installed qpid python. I've >> added new install instructions to the python tree. >> >> If you have a chance, and you think you may be affected, please take a >> look at the new state of things and let me know if there's any trouble. >> I'll be working on addressing problems today while the dust settles. >> >> Justin >> >> On Mon, Apr 18, 2016 at 9:44 AM, Justin Ross >> wrote: >> >>> On Mon, Apr 18, 2016 at 9:00 AM, Robbie Gemmell < >>> robbie.gemm...@gmail.com> wrote: Looks good. Again, goes much further than I was originally expecting initially :) I'm not sure I would copy the specs dir though, at least not until some future point if particular need arises, since little/nothing will really reference the copy. >>> >>> Okay, I'll hold off on that. >>> >>> The readme for the update no longer mentions the Java QMF bits, suggesting they aren't getting moved, but I see they are still in the reorg fork at a previously-moved-there location, so just in case: if nobody is committing to update and release them, as seems to be the case currently, then I'd also leave them where they are in the current repo until cause arises to do otherwise. >>> >>> Yes. It's been ambiguous in my mind as well as in the proposal. I will >>> leave them as-is for now and wait for a response from Fraser. After a >>> time, if I don't hear from him, I'll proceed with removing them in a >>> second-round cleanup. >>> >>> The above said, perhaps their current nested structure is the main reason they were moved in the fork, in which case perhaps removing them from trunk first is the way to go. Ditto the WCF bits. Might be best to create a pre-reorg tag of everything before commencing. >>> >>> Definitely. Good idea. >>> >>> Finally, I'd suggest to use svn directly to do the significant moves/copies, as using git-svn for example sometimes wont end up doing exactly what you wanted/expected in those cases. >>> >>> Will do. Thanks for the feedback and guidance. >>> >>> I've made a dry run and met with success, so I will kick this off >>> shortly. >>> >>> Justin >>> >> >> >
Re: Qpid source code reorg update
Hi, everyone. I'm now in the midst of getting the various CI environments working, adding some temporary valgrind suppressions and fixing incompatibilities with various versions of the buildchain tools. After CI is in suitably good shape, I would like to start the process to migrate qpid-cpp and qpid-python to git. That will require a vote, which I will kick off shortly. On Thu, Apr 21, 2016 at 7:06 AM, Justin Ross wrote: > I've committed the major pieces of this, and I'm now doing more testing > and improving the documentation. > > These changes are likely going to affect CI and test configurations. In > particular, the cpp tree now depends on an installed qpid python. I've > added new install instructions to the python tree. > > If you have a chance, and you think you may be affected, please take a > look at the new state of things and let me know if there's any trouble. > I'll be working on addressing problems today while the dust settles. > > Justin > > On Mon, Apr 18, 2016 at 9:44 AM, Justin Ross > wrote: > >> On Mon, Apr 18, 2016 at 9:00 AM, Robbie Gemmell > > wrote: >>> >>> Looks good. Again, goes much further than I was originally expecting >>> initially :) >>> >>> I'm not sure I would copy the specs dir though, at least not until >>> some future point if particular need arises, since little/nothing will >>> really reference the copy. >>> >> >> Okay, I'll hold off on that. >> >> >>> The readme for the update no longer mentions the Java QMF bits, >>> suggesting they aren't getting moved, but I see they are still in the >>> reorg fork at a previously-moved-there location, so just in case: if >>> nobody is committing to update and release them, as seems to be the >>> case currently, then I'd also leave them where they are in the current >>> repo until cause arises to do otherwise. >>> >> >> Yes. It's been ambiguous in my mind as well as in the proposal. I will >> leave them as-is for now and wait for a response from Fraser. After a >> time, if I don't hear from him, I'll proceed with removing them in a >> second-round cleanup. >> >> >>> The above said, perhaps their current nested structure is the main >>> reason they were moved in the fork, in which case perhaps removing >>> them from trunk first is the way to go. Ditto the WCF bits. >>> >>> Might be best to create a pre-reorg tag of everything before commencing. >>> >> >> Definitely. Good idea. >> >> >>> Finally, I'd suggest to use svn directly to do the significant >>> moves/copies, as using git-svn for example sometimes wont end up doing >>> exactly what you wanted/expected in those cases. >>> >> >> Will do. Thanks for the feedback and guidance. >> >> I've made a dry run and met with success, so I will kick this off shortly. >> >> Justin >> > >
Re: Qpid source code reorg update
I've committed the major pieces of this, and I'm now doing more testing and improving the documentation. These changes are likely going to affect CI and test configurations. In particular, the cpp tree now depends on an installed qpid python. I've added new install instructions to the python tree. If you have a chance, and you think you may be affected, please take a look at the new state of things and let me know if there's any trouble. I'll be working on addressing problems today while the dust settles. Justin On Mon, Apr 18, 2016 at 9:44 AM, Justin Ross wrote: > On Mon, Apr 18, 2016 at 9:00 AM, Robbie Gemmell > wrote: >> >> Looks good. Again, goes much further than I was originally expecting >> initially :) >> >> I'm not sure I would copy the specs dir though, at least not until >> some future point if particular need arises, since little/nothing will >> really reference the copy. >> > > Okay, I'll hold off on that. > > >> The readme for the update no longer mentions the Java QMF bits, >> suggesting they aren't getting moved, but I see they are still in the >> reorg fork at a previously-moved-there location, so just in case: if >> nobody is committing to update and release them, as seems to be the >> case currently, then I'd also leave them where they are in the current >> repo until cause arises to do otherwise. >> > > Yes. It's been ambiguous in my mind as well as in the proposal. I will > leave them as-is for now and wait for a response from Fraser. After a > time, if I don't hear from him, I'll proceed with removing them in a > second-round cleanup. > > >> The above said, perhaps their current nested structure is the main >> reason they were moved in the fork, in which case perhaps removing >> them from trunk first is the way to go. Ditto the WCF bits. >> >> Might be best to create a pre-reorg tag of everything before commencing. >> > > Definitely. Good idea. > > >> Finally, I'd suggest to use svn directly to do the significant >> moves/copies, as using git-svn for example sometimes wont end up doing >> exactly what you wanted/expected in those cases. >> > > Will do. Thanks for the feedback and guidance. > > I've made a dry run and met with success, so I will kick this off shortly. > > Justin >
Re: Qpid source code reorg update
On Mon, Apr 18, 2016 at 9:00 AM, Robbie Gemmell wrote: > > Looks good. Again, goes much further than I was originally expecting > initially :) > > I'm not sure I would copy the specs dir though, at least not until > some future point if particular need arises, since little/nothing will > really reference the copy. > Okay, I'll hold off on that. > The readme for the update no longer mentions the Java QMF bits, > suggesting they aren't getting moved, but I see they are still in the > reorg fork at a previously-moved-there location, so just in case: if > nobody is committing to update and release them, as seems to be the > case currently, then I'd also leave them where they are in the current > repo until cause arises to do otherwise. > Yes. It's been ambiguous in my mind as well as in the proposal. I will leave them as-is for now and wait for a response from Fraser. After a time, if I don't hear from him, I'll proceed with removing them in a second-round cleanup. > The above said, perhaps their current nested structure is the main > reason they were moved in the fork, in which case perhaps removing > them from trunk first is the way to go. Ditto the WCF bits. > > Might be best to create a pre-reorg tag of everything before commencing. > Definitely. Good idea. > Finally, I'd suggest to use svn directly to do the significant > moves/copies, as using git-svn for example sometimes wont end up doing > exactly what you wanted/expected in those cases. > Will do. Thanks for the feedback and guidance. I've made a dry run and met with success, so I will kick this off shortly. Justin
Re: Qpid source code reorg update
On 7 April 2016 at 01:02, Justin Ross wrote: > Proposal: https://github.com/ssorj/qpid-svn-reorg > Previous discussion: > http://qpid.2158936.n2.nabble.com/Qpid-Subversion-reorganization-proposal-td7639094.html > > Hi, everyone. > > Since my last update, I have been trying to get the revamped C++ tests to > function under Windows. I've had only mixed success, but now I believe > it's time to press on. > > To recap, in addition to reorganizing the source modules for independent > releases, my work overhauled the C++ tests. This means that there is now > greater potential for Qpid C++ testing on Windows, but that potential will > require more work to realize. > > You can see some representative output here: > > https://ci.appveyor.com/project/ssorj/qpid-svn-reorg/build/trunk.66 > > High level summary: the unit tests are not working[*] (and perhaps most of > all, I'd like go get *them* working), but many of the other tests are > working as expected. I am satisfied that most of the infrastructure for > making those tests work is in place, and we can start to tackle these > things as individual test issues, not test suite design issues. > > On Linux (or rather on my F23 box), the tests are in relatively good > shape. I don't believe the test situation wrt Windows is a regression. > Rather, even though there are many problems to address, I think it may be a > net improvement. > > Aside: there have been some related usability improvements as well. The > install mechanisms now include batch files so that important tools such as > qpid-config and qpid-python-test are executable from the Windows command > line. > > What's next? I'm now up against the scheduled release of Proton 0.13.0. > Here's the sequence I have in mind: > > - Merge my changes to "trunk" (see also > https://github.com/ssorj/qpid-svn-reorg#sequence-with-respect-to-git-migration > ) > - Start the Proton 0.13.0 release process > - Meanwhile, address new issues arising from the reorg > - Complete the Proton 0.13.0 release > - Start and complete the Qpid C++ 1.35.0 and Qpid Python 1.35.0 releases, > using Proton 0.13.0 as a tested dependency > > Thanks, > Justin > > --- > > [*] In the appveyor output above, the part where the unit tests run, you > see a failure about not finding the appveyor-vm cert. Qpidd on Windows by > default tries to find a cert in CurrentUser/My. The PowerShell scripting I > use to set up the appveyor cert can be adapted to add one there, but it > produces a dialog box warning that I have not yet been able to suppress > (grumble). I have also tried to configure the unit tests to load the cert > from LocalMachine/My using QPID_* environment variables. While those > variables have an effect on the daemon, they do not seem to have any effect > on the unit tests binary. > > In any case, when I run the unit tests on a local windows machine where I > can setup the certs correctly, I see another and more cryptic failure. > > Some related pieces. Please let me know if you have suggestions. I would > really like to make this work under appveyor. > > https://github.com/ssorj/qpid-svn-reorg/blob/trunk/appveyor.yml#L40 > https://github.com/ssorj/qpid-svn-reorg/blob/trunk/scripts/InstallSelfSignedCert.ps1 > https://github.com/ssorj/qpid-svn-reorg/blob/trunk/qpid/cpp/src/tests/run_unit_tests#L32 Looks good. Again, goes much further than I was originally expecting initially :) I'm not sure I would copy the specs dir though, at least not until some future point if particular need arises, since little/nothing will really reference the copy. The readme for the update no longer mentions the Java QMF bits, suggesting they aren't getting moved, but I see they are still in the reorg fork at a previously-moved-there location, so just in case: if nobody is committing to update and release them, as seems to be the case currently, then I'd also leave them where they are in the current repo until cause arises to do otherwise. The above said, perhaps their current nested structure is the main reason they were moved in the fork, in which case perhaps removing them from trunk first is the way to go. Ditto the WCF bits. Might be best to create a pre-reorg tag of everything before commencing. Finally, I'd suggest to use svn directly to do the significant moves/copies, as using git-svn for example sometimes wont end up doing exactly what you wanted/expected in those cases. Robbie - To unsubscribe, e-mail: users-unsubscr...@qpid.apache.org For additional commands, e-mail: users-h...@qpid.apache.org
Qpid source code reorg update
Proposal: https://github.com/ssorj/qpid-svn-reorg Previous discussion: http://qpid.2158936.n2.nabble.com/Qpid-Subversion-reorganization-proposal-td7639094.html Hi, everyone. Since my last update, I have been trying to get the revamped C++ tests to function under Windows. I've had only mixed success, but now I believe it's time to press on. To recap, in addition to reorganizing the source modules for independent releases, my work overhauled the C++ tests. This means that there is now greater potential for Qpid C++ testing on Windows, but that potential will require more work to realize. You can see some representative output here: https://ci.appveyor.com/project/ssorj/qpid-svn-reorg/build/trunk.66 High level summary: the unit tests are not working[*] (and perhaps most of all, I'd like go get *them* working), but many of the other tests are working as expected. I am satisfied that most of the infrastructure for making those tests work is in place, and we can start to tackle these things as individual test issues, not test suite design issues. On Linux (or rather on my F23 box), the tests are in relatively good shape. I don't believe the test situation wrt Windows is a regression. Rather, even though there are many problems to address, I think it may be a net improvement. Aside: there have been some related usability improvements as well. The install mechanisms now include batch files so that important tools such as qpid-config and qpid-python-test are executable from the Windows command line. What's next? I'm now up against the scheduled release of Proton 0.13.0. Here's the sequence I have in mind: - Merge my changes to "trunk" (see also https://github.com/ssorj/qpid-svn-reorg#sequence-with-respect-to-git-migration ) - Start the Proton 0.13.0 release process - Meanwhile, address new issues arising from the reorg - Complete the Proton 0.13.0 release - Start and complete the Qpid C++ 1.35.0 and Qpid Python 1.35.0 releases, using Proton 0.13.0 as a tested dependency Thanks, Justin --- [*] In the appveyor output above, the part where the unit tests run, you see a failure about not finding the appveyor-vm cert. Qpidd on Windows by default tries to find a cert in CurrentUser/My. The PowerShell scripting I use to set up the appveyor cert can be adapted to add one there, but it produces a dialog box warning that I have not yet been able to suppress (grumble). I have also tried to configure the unit tests to load the cert from LocalMachine/My using QPID_* environment variables. While those variables have an effect on the daemon, they do not seem to have any effect on the unit tests binary. In any case, when I run the unit tests on a local windows machine where I can setup the certs correctly, I see another and more cryptic failure. Some related pieces. Please let me know if you have suggestions. I would really like to make this work under appveyor. https://github.com/ssorj/qpid-svn-reorg/blob/trunk/appveyor.yml#L40 https://github.com/ssorj/qpid-svn-reorg/blob/trunk/scripts/InstallSelfSignedCert.ps1 https://github.com/ssorj/qpid-svn-reorg/blob/trunk/qpid/cpp/src/tests/run_unit_tests#L32
