Re: Why do we create the cpp/src/tests/python dir?
Alan Conway wrote: The C++ builds install python files in src/tests/python. This is kind of a pain for at least 2 reasons: - src/tests/python can go stale - not sure how but it just happened to me. - debugging python code, error messages take you to the src/tests/python copy rather than the real source file. - python gets re-installed every time you build. Whats the reason for doing this? Can we get the same effect some other way that doesn't involve duplicating all the .py files? The root cause for this is that the python client needs to support python 2.3, and to do this it needs to do some preprocessing on the files before you can run them (assuming you're on a system with python 2.3). So it's not actually copying the files it's actually compiling them, and the reason this is done under the cpp build root is simply to not pollute the source tree. As for why they get re-installed every time and how they can go stale, I'm not sure about that. I believe andrew was the one who modified the cpp build to do this. I'm guessing it could be done in a smarter way that wouldn't rebuild and wouldn't allow things to go stale. --Rafael - Apache Qpid - AMQP Messaging Implementation Project: http://qpid.apache.org Use/Interact: mailto:dev-subscr...@qpid.apache.org
Re: Why do we create the cpp/src/tests/python dir?
On 03/19/2010 07:59 AM, Rafael Schloming wrote: Alan Conway wrote: The C++ builds install python files in src/tests/python. This is kind of a pain for at least 2 reasons: - src/tests/python can go stale - not sure how but it just happened to me. - debugging python code, error messages take you to the src/tests/python copy rather than the real source file. - python gets re-installed every time you build. Whats the reason for doing this? Can we get the same effect some other way that doesn't involve duplicating all the .py files? The root cause for this is that the python client needs to support python 2.3, and to do this it needs to do some preprocessing on the files before you can run them (assuming you're on a system with python 2.3). So it's not actually copying the files it's actually compiling them, and the reason this is done under the cpp build root is simply to not pollute the source tree. As for why they get re-installed every time and how they can go stale, I'm not sure about that. I believe andrew was the one who modified the cpp build to do this. I'm guessing it could be done in a smarter way that wouldn't rebuild and wouldn't allow things to go stale. I think this is a good case study to think about how we do dependencies between sub-projects in qpid. The C++ build currently has dependencies on (at least) - python - tools - extras And it picks them up via a variety of routes. In an svn checkout it has a bunch of coded-in assumptions about looking for things in ../python, it's got hackery to run the python install process mentioned above and it's all quite nasty. In a distribution tree it picks up qpid-tool and the like from PATH if they're installed, quietly skips the python tests if it can't find the ../python tree, randomly fails if there happens to be a ../python directory that's not what the build expected etc. I think the clean way out is to make the python project a dependency of the C++ project just like it depends on boost or other tools. I.e. to run the tests you must *install* the python project, and the tests pick up components via standard PATH and PYTHONPATH env vars. A developer would set their install --prefix to their own directory rather than the system one and install the dependencies there. A user building from a distro could do this or install in standard system places as they wish. To keep things simple for developers we would have a script that install all the C++ deps for you in an SVN checkout. This idea is in the half-baked stage so please point to holes suggest improvements. There are a couple drawbacks I can think of: - extra step for developers to build, although could consider this part of installing pre-reqs so not really a new step. - risk of picking up stuff installed in standard places by accident, getting wrong version. Don't know how to avoid this. - not sure how this plays in the windows world. Thoughts? Alan. - Apache Qpid - AMQP Messaging Implementation Project: http://qpid.apache.org Use/Interact: mailto:dev-subscr...@qpid.apache.org
Re: Why do we create the cpp/src/tests/python dir?
On Fri, 2010-03-19 at 07:59 -0400, Rafael Schloming wrote: As for why they get re-installed every time and how they can go stale, I'm not sure about that. I believe andrew was the one who modified the cpp build to do this. I plead not guilty m'lud. - Apache Qpid - AMQP Messaging Implementation Project: http://qpid.apache.org Use/Interact: mailto:dev-subscr...@qpid.apache.org
Re: Why do we create the cpp/src/tests/python dir?
Gordon Sim wrote: On 03/19/2010 02:23 PM, Alan Conway wrote: I think the clean way out is to make the python project a dependency of the C++ project just like it depends on boost or other tools. I.e. to run the tests you must *install* the python project, and the tests pick up components via standard PATH and PYTHONPATH env vars. Assuming I understand this correctly, we would have to reinstall every time we needed to pick up any change in the python client or tools. Unlike with boost or other tools, that's likely to occur quite frequently. I'm not arguing against your suggestion however, just making an observation on the impact as I see it. There is also a dependency on tests, which would probably be more problematic since it's not uncommon to fix/add something to the broker and also change these tests in a single commit. --Rafael - Apache Qpid - AMQP Messaging Implementation Project: http://qpid.apache.org Use/Interact: mailto:dev-subscr...@qpid.apache.org
Why do we create the cpp/src/tests/python dir?
The C++ builds install python files in src/tests/python. This is kind of a pain for at least 2 reasons: - src/tests/python can go stale - not sure how but it just happened to me. - debugging python code, error messages take you to the src/tests/python copy rather than the real source file. - python gets re-installed every time you build. Whats the reason for doing this? Can we get the same effect some other way that doesn't involve duplicating all the .py files? - Apache Qpid - AMQP Messaging Implementation Project: http://qpid.apache.org Use/Interact: mailto:dev-subscr...@qpid.apache.org
