Re: The burning technical issues of the day
Anyone one with time and some experience with this tool is welcome to go ahead and explore, as far as I am concerned. Karl On Wed, May 18, 2011 at 4:57 AM, Tommaso Teofili tommaso.teof...@gmail.com wrote: 2011/5/17 Koji Sekiguchi k...@r.email.ne.jp I've been thinking mock for unit test, too. Does anyone know there are any projects that connect proprietary software in Apache? We can consult them, if any. Tommaso, do you know Alchemy Annotator uses mock for the test in UIMA? Yes, I wrote it :-) That is a basic mocking as there is a clear API for the alchemyapi.comwebservice and I only wrote an implementation returning a predefined XML response, so nothing special actually. In the past I remember I used Selenium [1] to write mocked integration tests while writing a proxy but I remember that was hard to maintain in a changing environment. Let's keep the discussion and ideas flow :-) Tommaso [1] : http://seleniumhq.org/ Koji -- http://www.rondhuit.com/en/ (11/05/17 22:10), Tommaso Teofili wrote: Hello Karl, thanks so much again for the warm welcome and for these insights. I agree with you that testability is a major concern we need to engage and get sorted. One thing I think right now is that having tests rely on a testing infrastructure would possibly expose ManifoldCF to this issue again in case of maintenance, versions evolution and so on therefore, although I still have to investigate if that can be sorted technically for the supported systems, my design time opinion is that we should try to mock those systems. Did you try to mock any of them yet? Does this sound good/bad to you? Regards, Tommaso 2011/5/17 Karl Wrightdaddy...@gmail.com For those who have just entered the ManifoldCF project, I'd like to first extend my congratulations once again! You are probably still trying to figure out exactly what's going on and where we are going. Unfortunately, this being Apache, I cannot actually answer your question, because you are part of the process now, and you will now be able to act on your own ideas and goals for the project. But in the interests of planning and consensus building, I'd like to share my thoughts as to what I think are the major technical issues the project faces. The first and foremost issue is one of testability. ManifoldCF is unique in that every connector requires a system to test against. In some cases this is a proprietary system, such as Documentum or LiveLink. In other cases, it's a large number of web servers. When MetaCarta developed the project in the first place, we had a stable of VMs against which our tests worked. They had proprietary software installed in some cases, or some Apache hackery to emulate a large number of individual web sites. When MetaCarta assets were sold to qBase, those assets included those VMs. We desperately need something like this again. I've tried to get access to the old MetaCarta VM's, but qBase has not yet granted this, and may never grant it due to ongoing technical reasons. Given this, it seems that we need some way of doing the same thing. It would be great to hear your ideas, especially if you have access to any of the proprietary systems we support and would be willing to set up and maintain testing infrastructure. Again, welcome to our new committers and new mentor! Karl
Re: The burning technical issues of the day
2011/5/17 Koji Sekiguchi k...@r.email.ne.jp I've been thinking mock for unit test, too. Does anyone know there are any projects that connect proprietary software in Apache? We can consult them, if any. Tommaso, do you know Alchemy Annotator uses mock for the test in UIMA? Yes, I wrote it :-) That is a basic mocking as there is a clear API for the alchemyapi.comwebservice and I only wrote an implementation returning a predefined XML response, so nothing special actually. In the past I remember I used Selenium [1] to write mocked integration tests while writing a proxy but I remember that was hard to maintain in a changing environment. Let's keep the discussion and ideas flow :-) Tommaso [1] : http://seleniumhq.org/ Koji -- http://www.rondhuit.com/en/ (11/05/17 22:10), Tommaso Teofili wrote: Hello Karl, thanks so much again for the warm welcome and for these insights. I agree with you that testability is a major concern we need to engage and get sorted. One thing I think right now is that having tests rely on a testing infrastructure would possibly expose ManifoldCF to this issue again in case of maintenance, versions evolution and so on therefore, although I still have to investigate if that can be sorted technically for the supported systems, my design time opinion is that we should try to mock those systems. Did you try to mock any of them yet? Does this sound good/bad to you? Regards, Tommaso 2011/5/17 Karl Wrightdaddy...@gmail.com For those who have just entered the ManifoldCF project, I'd like to first extend my congratulations once again! You are probably still trying to figure out exactly what's going on and where we are going. Unfortunately, this being Apache, I cannot actually answer your question, because you are part of the process now, and you will now be able to act on your own ideas and goals for the project. But in the interests of planning and consensus building, I'd like to share my thoughts as to what I think are the major technical issues the project faces. The first and foremost issue is one of testability. ManifoldCF is unique in that every connector requires a system to test against. In some cases this is a proprietary system, such as Documentum or LiveLink. In other cases, it's a large number of web servers. When MetaCarta developed the project in the first place, we had a stable of VMs against which our tests worked. They had proprietary software installed in some cases, or some Apache hackery to emulate a large number of individual web sites. When MetaCarta assets were sold to qBase, those assets included those VMs. We desperately need something like this again. I've tried to get access to the old MetaCarta VM's, but qBase has not yet granted this, and may never grant it due to ongoing technical reasons. Given this, it seems that we need some way of doing the same thing. It would be great to hear your ideas, especially if you have access to any of the proprietary systems we support and would be willing to set up and maintain testing infrastructure. Again, welcome to our new committers and new mentor! Karl
Re: The burning technical issues of the day
Hello Karl, thanks so much again for the warm welcome and for these insights. I agree with you that testability is a major concern we need to engage and get sorted. One thing I think right now is that having tests rely on a testing infrastructure would possibly expose ManifoldCF to this issue again in case of maintenance, versions evolution and so on therefore, although I still have to investigate if that can be sorted technically for the supported systems, my design time opinion is that we should try to mock those systems. Did you try to mock any of them yet? Does this sound good/bad to you? Regards, Tommaso 2011/5/17 Karl Wright daddy...@gmail.com For those who have just entered the ManifoldCF project, I'd like to first extend my congratulations once again! You are probably still trying to figure out exactly what's going on and where we are going. Unfortunately, this being Apache, I cannot actually answer your question, because you are part of the process now, and you will now be able to act on your own ideas and goals for the project. But in the interests of planning and consensus building, I'd like to share my thoughts as to what I think are the major technical issues the project faces. The first and foremost issue is one of testability. ManifoldCF is unique in that every connector requires a system to test against. In some cases this is a proprietary system, such as Documentum or LiveLink. In other cases, it's a large number of web servers. When MetaCarta developed the project in the first place, we had a stable of VMs against which our tests worked. They had proprietary software installed in some cases, or some Apache hackery to emulate a large number of individual web sites. When MetaCarta assets were sold to qBase, those assets included those VMs. We desperately need something like this again. I've tried to get access to the old MetaCarta VM's, but qBase has not yet granted this, and may never grant it due to ongoing technical reasons. Given this, it seems that we need some way of doing the same thing. It would be great to hear your ideas, especially if you have access to any of the proprietary systems we support and would be willing to set up and maintain testing infrastructure. Again, welcome to our new committers and new mentor! Karl
Re: The burning technical issues of the day
Hi Tommaso, Mocking up a third-party system is preferable to having no runnable tests at all, and it would likely prevent us from introducing regressions in the code. On the other hand, such an approach does not support forward development or configuration exploration. As you point out, these systems release new versions periodically, and no doubt people will eventually want to add features to the existing connectors, or support different configurations, for which a mock-up approach will not be sufficient. So I don't think mock-ups will be a complete replacement for end-to-end testing against an actual instance, although as I said, it would be better than nothing. This logic applies most completely to: - FileNet - Documentum - LiveLink - Meridio - SharePoint Other connectors, such as the web connector and rss connector, are more difficult to test via mockup and need instead something to crawl against. We used a DNS fakeout trick at MetaCarta - basically, messing with the DNS setup to point all servers at the same box, but that's hard to do under Java because it caches DNS, and tests that mess with local DNS settings would probably upset some people who ran them unknowingly. ;-) Karl On Tue, May 17, 2011 at 9:10 AM, Tommaso Teofili tommaso.teof...@gmail.com wrote: Hello Karl, thanks so much again for the warm welcome and for these insights. I agree with you that testability is a major concern we need to engage and get sorted. One thing I think right now is that having tests rely on a testing infrastructure would possibly expose ManifoldCF to this issue again in case of maintenance, versions evolution and so on therefore, although I still have to investigate if that can be sorted technically for the supported systems, my design time opinion is that we should try to mock those systems. Did you try to mock any of them yet? Does this sound good/bad to you? Regards, Tommaso 2011/5/17 Karl Wright daddy...@gmail.com For those who have just entered the ManifoldCF project, I'd like to first extend my congratulations once again! You are probably still trying to figure out exactly what's going on and where we are going. Unfortunately, this being Apache, I cannot actually answer your question, because you are part of the process now, and you will now be able to act on your own ideas and goals for the project. But in the interests of planning and consensus building, I'd like to share my thoughts as to what I think are the major technical issues the project faces. The first and foremost issue is one of testability. ManifoldCF is unique in that every connector requires a system to test against. In some cases this is a proprietary system, such as Documentum or LiveLink. In other cases, it's a large number of web servers. When MetaCarta developed the project in the first place, we had a stable of VMs against which our tests worked. They had proprietary software installed in some cases, or some Apache hackery to emulate a large number of individual web sites. When MetaCarta assets were sold to qBase, those assets included those VMs. We desperately need something like this again. I've tried to get access to the old MetaCarta VM's, but qBase has not yet granted this, and may never grant it due to ongoing technical reasons. Given this, it seems that we need some way of doing the same thing. It would be great to hear your ideas, especially if you have access to any of the proprietary systems we support and would be willing to set up and maintain testing infrastructure. Again, welcome to our new committers and new mentor! Karl
Re: The burning technical issues of the day
I've been thinking mock for unit test, too. Does anyone know there are any projects that connect proprietary software in Apache? We can consult them, if any. Tommaso, do you know Alchemy Annotator uses mock for the test in UIMA? Koji -- http://www.rondhuit.com/en/ (11/05/17 22:10), Tommaso Teofili wrote: Hello Karl, thanks so much again for the warm welcome and for these insights. I agree with you that testability is a major concern we need to engage and get sorted. One thing I think right now is that having tests rely on a testing infrastructure would possibly expose ManifoldCF to this issue again in case of maintenance, versions evolution and so on therefore, although I still have to investigate if that can be sorted technically for the supported systems, my design time opinion is that we should try to mock those systems. Did you try to mock any of them yet? Does this sound good/bad to you? Regards, Tommaso 2011/5/17 Karl Wrightdaddy...@gmail.com For those who have just entered the ManifoldCF project, I'd like to first extend my congratulations once again! You are probably still trying to figure out exactly what's going on and where we are going. Unfortunately, this being Apache, I cannot actually answer your question, because you are part of the process now, and you will now be able to act on your own ideas and goals for the project. But in the interests of planning and consensus building, I'd like to share my thoughts as to what I think are the major technical issues the project faces. The first and foremost issue is one of testability. ManifoldCF is unique in that every connector requires a system to test against. In some cases this is a proprietary system, such as Documentum or LiveLink. In other cases, it's a large number of web servers. When MetaCarta developed the project in the first place, we had a stable of VMs against which our tests worked. They had proprietary software installed in some cases, or some Apache hackery to emulate a large number of individual web sites. When MetaCarta assets were sold to qBase, those assets included those VMs. We desperately need something like this again. I've tried to get access to the old MetaCarta VM's, but qBase has not yet granted this, and may never grant it due to ongoing technical reasons. Given this, it seems that we need some way of doing the same thing. It would be great to hear your ideas, especially if you have access to any of the proprietary systems we support and would be willing to set up and maintain testing infrastructure. Again, welcome to our new committers and new mentor! Karl
