Hi Justin, This actually goes quite far beyond what I was expecting before the upcoming cpp etc releases, looks like you have been busy! :)
Just to confirm, is the thinking that the current heirarchy in https://svn.apache.org/repos/asf/qpid/trunk/qpid/ dissappear entirely, with most of the resulting subdirs of the 'qpid-svn-reorg/qpid/' on the github repo effectively being added as new svn trunk+branches+tags heirarchies alongside the remaining dirs currently at https://svn.apache.org/repos/asf/qpid? Thats how it mostly reads, just wanted to cofirm since some paths shown in the readme dont quite line up with that. Mentioning git...when the java bits were looking to move within svn, discussion and testing around any subsequent future moves to git suggested there will be some issues keeping the full history of commits for a given component in the resulting git repos due to all the moves. As this would effectively move all of the current code, I guess that could make it harder still. However, given the svn repo and history will stay in place (as it has for the other compoents that migrated), and these bits may never migrate to git repos, this isnt necessarily an issue but just something to note for consideration. Beyond the above, I had some component-specific thoughts/questions: # WCF These bits havent actually been getting released of late (not since 0.28 in May 2014, though it is in the subsequent tags) or even updated (not since Nov 2012), so should we bother to move them? Perhaps even time for a different discussion about them? # Saslwrapper If it is only [optionally] used by the old python client as suggested, should it perhaps be in the tree with that? It doesnt seem to have been touched in 5 years, or released at all in the last couple. Splitting it out isnt so clear to me. # Java QMF tools To your note around whether it may make sense to add these into qpid-java, I think them remaining separate seems perferable. Adding them in goes against the grain of making the java bits more independently releasable, with further modularisation of the existing grouped bits already a possibility in future. The tools were also written primarily against the C++ broker given its reliance on QMF for management and although a QMF broker plugin was added for the java broker later that allowed the tools to work to an extent against both brokers, that doesnt look to be keeping up. Other than some trivial removals a year ago to match the python tools, there havent been any changes for 18 months. In looking at when they were last released (0.32 a year ago, which [almost] explains the versions still being 0.32-SNAPSHOT in svn), I came to realise they aren't actually listed on the website. I think it makes sense to leave these bits separate, or possibly even have a different discussion about them too. Robbie On 24 February 2016 at 04:06, Justin Ross <justin.r...@gmail.com> wrote: > Hi, everyone. I've been preparing a proof of concept for the reorg, and > it's reached a stage where it's ready for comment. > > https://github.com/ssorj/qpid-svn-reorg > > Look here for an idea of how the Qpid subversion top level would appear > after these changes: > > https://github.com/ssorj/qpid-svn-reorg/tree/trunk/qpid > > I have remaining work to get the C++ tests in shape, but the overall form > of things is what I wish to propose. I would appreciate your comments. > This is a functioning alternate source organization, so you can download it > and test it directly. > > So you can make comments inline, I've appended the markdown README below. > I will produce revised proposals after incorporating your input. > > Thanks, > Justin > > --- > > # Qpid Subversion reorganization > > ## Overview > > This proof of concept represents an effort to achieve the source tree > layout proposed [here][1]. It allows the Qpid project to produce > independent releases of Qpid C++ and Python as well as other modules > that have heretofore been bundled into one large Qpid release. > > [1]: > https://cwiki.apache.org/confluence/display/qpid/Source+tree+layout+proposal > > ### End-goal top-level codebases and source artifacts > > New at the top level > > qpid-cpp qpid/cpp/trunk qpid-cpp-VERSION.tar.gz > qpid-java-qmf qpid/java-qmf/trunk qpid-java-qmf-VERSION.tar.gz > qpid-python qpid/python/trunk qpid-python-VERSION.tar.gz > qpid-wcf qpid/wcf/trunk qpid-wcf-VERSION.tar.gz > saslwrapper qpid/saslwrapper/trunk saslwrapper-VERSION.tar.gz > > Existing > > qpid-java qpid/java/trunk qpid-java-VERSION.tar.gz > > Existing and migrated to Git > > qpid-dispatch qpid-dispatch.git qpid-dispatch-VERSION.tar.gz > qpid-jms qpid-jms.git apache-qpid-jms-VERSION.tar.gz > qpid-proton qpid-proton.git qpid-proton-VERSION.tar.gz > > No source releases > > qpid-maven qpid/maven/trunk [NA] > qpid-site qpid/site [NA] > qpid-specs qpid/specs [NA] > > In the current proposal, qpid-java-qmf stands apart from qpid-java, > but it may make more sense for java-qmf to take a place inside of > qpid-java. > > ## Dependencies > > qpid-cpp depends on qpid-proton for amqp 1.0 support > qpid-cpp depends on qpid-python for its tests > qpid-dispatch depends on qpid-proton > qpid-java depends on qpid-python for its tests > qpid-java-qmf depends on qpid-java > qpid-jms depends on qpid-proton > qpid-python optionally depends on saslwrapper > qpid-wcf depends on qpid-cpp > > ## Changes in the Subversion tree > > Note that the proof of concept does not in fact move the new top-level > codebases into the standard Subversion structure with trunk, branches, > and tags. In order to minimize diffs, they instead remain in their > current locations, but updated to remove cross-tree source > dependencies. > > ### Relocations > > Migrate qpid/trunk/qpid/tools python and ruby content to > qpid/cpp/management: > > qpid/trunk/tools/setup.py -> > qpid/cpp/trunk/management/setup.py > qpid/trunk/tools/MANIFEST.in -> > qpid/cpp/trunk/management/MANIFEST.in > qpid/trunk/tools/*.txt -> > qpid/cpp/trunk/management/python/*.txt > qpid/trunk/tools/src/py/$libs -> > qpid/cpp/trunk/management/python/lib/$libs > qpid/trunk/extras/qmf/src/py/qmf -> > qpid/cpp/trunk/management/python/lib/qmf > qpid/trunk/tools/src/py/$tools -> > qpid/cpp/trunk/management/python/bin/$tools > qpid/trunk/tools/src/ruby/qpid_management -> > qpid/cpp/trunk/management/ruby/qpid_management > > Migrate qpid/trunk/qpid/tools java content to qpid/java-qmf: > > qpid/trunk/tools/src/java -> qpid/java-qmf/trunk > > Migrate qpid/trunk/qpid/tests to qpid/python: > > qpid/trunk/tests/src/py/qpid_tests -> > qpid/python/trunk/qpid_tests > > Migrate Windows packaging scripts to qpid/cpp: > > qpid/trunk/qpid/packaging/windows -> > qpid/cpp/trunk/packaging/windows > > Migrate other self-contained components to the top level: > > qpid/trunk/qpid/extras/sasl -> qpid/saslwrapper/trunk > qpid/trunk/qpid/wcf -> qpid/wcf/trunk > > ### Deletions > > Obsolete: > > qpid/trunk/qpid/QPID_VERSION.txt # No longer makes sense > qpid/trunk/qpid/LICENSE # No longer makes sense > qpid/trunk/qpid/NOTICE # No longer makes sense > qpid/trunk/qpid/extras/dispatch # Obsolete stub > qpid/trunk/qpid/doc/website # Obsolete stub > > After migration: > > qpid/trunk/qpid/doc/book # Moved under > qpid/cpp/trunk/docs > qpid/trunk/qpid/extras/qmf # Moved under > qpid/cpp/trunk/management/python > > Apparently no longer in use. I'm judging "in use" by whether there > have been any changes--not an excellent metric. If you are using any > of these, please inform me so that I preserve them. > > qpid/trunk/qpid/review # Not touched since 2009 > qpid/trunk/qpid/sandbox # Not touched since 2010 > qpid/trunk/qpid/bin # Not touched since 2012 > qpid/trunk/qpid/buildtools # Not touched since 2010 > qpid/trunk/qpid/doc/dev-readme # Not touched since 2010 > qpid/trunk/qpid/etc # Not touched since 2008 > > ## C++ subtree changes > > Relocate some docs: > > cpp/design_docs -> cpp/docs/design > cpp/docs/src/* -> cpp/docs/design > cpp/DESIGN -> cpp/docs/design/overview.txt > > cpp/AMQP_1.0 -> cpp/docs/amqp-1.0.txt > cpp/SSL -> cpp/docs/ssl.txt > cpp/README-HA.txt -> cpp/docs/ha.txt > > Collect winsdk stuff: > > cpp/bld-winsdk.ps1 -> cpp/packaging/winsdk/bld-winsdk.ps1 > cpp/README-winsdk.ps1 -> cpp/packaging/winsdk/README.txt > > Clean up names and extensions: > > cpp/QPID_VERSION.txt -> cpp/VERSION.txt # Matching what Dispatch > has > cpp/NOTICE -> cpp/NOTICE.txt > cpp/LICENSE -> cpp/LICENSE.txt # Using .txt extension > for these > cpp/INSTALL -> cpp/INSTALL.txt # is widespread practice > now > cpp/INSTALL-WINDOWS -> cpp/INSTALL-WINDOWS.txt > > ## Possibly impacted parties > > - Steve Huston - Windows packaging code (qpid/packaging/windows) > - Java broker folks - Relocation of Python qpid_tests may affect Java > broker testing > > ## How to setup the Qpid Python dependency > > cd qpid/python/trunk > ./setup.py install --user > export PYTHONPATH=$HOME/.local/lib/python2.7/site-packages > export PATH=$HOME/.local/bin:$PATH > > ## C++ tests > > Changes: > > - Run scripts ("run_" scripts) organized by feature area > - Run scripts are uniformly named > - Run scripts produce temporary work dirs of the form run_script\_name_XXXX > - They are removed on test success > - All run scripts are runnable directly from src/tests build dir > - Python test env and test broker tools - should be cross platform > - Permits removal of many scripts that have both bash and > powershell variants > - Broker creation is more consistent > - Broker termination and checking too > - Modernized bash scripts > > Test results after initial relocation: > > The following tests FAILED: > 1 - api_check_qpidtypes (Failed) > 2 - api_check_qpidmessaging (Failed) > 20 - interop_tests (Failed) > 21 - ha_tests (Failed) > 22 - qpidd_qmfv2_tests (Failed) > 23 - interlink_tests (Failed) > 24 - idle_timeout_tests (Failed) > Errors while running CTest > Makefile:149: recipe for target 'test' failed > make: *** [test] Error 8 > > It was actually worse than this suggests. Lots of the tests silently > pass (!) if the python testing tools are not available. > > Current test results on Fedora 23: > > Total Test time (real) = 2053.14 sec > > The following tests FAILED: > 1 - api_check_qpidtypes (Failed) [C++11 abi changes] > 2 - api_check_qpidmessaging (Failed) [C++11 abi changes] > 4 - acl_tests (Failed) [valgrind error] > 12 - python_tests (Failed) ["amqp:precondition-failed: No > target specified!"] > 13 - queue_redirect_tests (Failed) [valgrind error] > 26 - linearstore_python_tests (Failed) [valgrind error] > Errors while running CTest > > ## Notes > > - Gah! Circular dependency. Qpid Python (via qpid.testlib) depends > on qpidtoollibs. This should be removed: Qpid C++ depends on Qpid > Python and not the reverse in any way. > > - The new test regime more uniformly runs brokers with valgrind, and > this exposes new valgrind errors. > > - I removed the following test install utils from src/tests: > qpid-build-rinstall, install_env.sh, etc. If these are still > important, I'd like to bring them back in a subdirectory or a > different project altogether. They are not central to the C++ > tests. > > ## Todo > > - Rewrite the index in qpid/README.txt - Update all the top-level > info > - Lots of tedious, infuriating Windows testing > - Current results at > https://ci.appveyor.com/project/ssorj/qpid-svn-reorg/ --------------------------------------------------------------------- To unsubscribe, e-mail: users-unsubscr...@qpid.apache.org For additional commands, e-mail: users-h...@qpid.apache.org
