Connector architecture question and suggestion
Hi, I am having a look at the connectors. At the moment to my opinion the classes for (all) connectors are to big. This is partly due to the way the interfaces are structured and partly due to the implementation of html in java. For example the RssConnector now has almost 6000 lines of code and the JdbcConnector has almost 2000 lines of code. To my opinion this can be improved by making separate components for presenting and configuring the connectors in the crawler-ui web project and for the part needed by the runner. Abstracting the html from the actual classes can help a lot as well. Maybe some utility methods to make creating these html pages easier is nice as well. I am willing to investigate this path further, but I'd like to have ideas of other developers. It would be nice to know if others feel like this can be improved as well. It might be interesting to look at a technique like wicket for the ui part. Than you can package the html code together with the java code in one jar. No difficult repackaging is required and you can still create nice interfaces. I also read that others want to have a look at something like velocity, of course that can be a valid option as well. So what is your opinions about it? regards Jettro Coenradie http://www.gridshore.nl
RE: Connector architecture question and suggestion
I think that providing tools/help for implementing the UI pieces of connectors is a perfectly reasonable thing to do. However, I strongly believe that the UI components should remain described as part of the connector interfaces. Breaking their implementations out within individual connectors is also reasonable, but unless there is a compelling reason to refactor all the individual connectors in that way, I would hold off that project until there are better unit tests for the connector UI components. If you are willing to contribute such tests, I think going that way would be worth consideration. I'd like to see a more detailed proposal before I comment further. Thanks, Karl -Original Message- From: jettro.coenra...@gmail.com [mailto:jettro.coenra...@gmail.com] On Behalf Of ext Jettro Coenradie Sent: Monday, August 16, 2010 7:37 AM To: connectors-dev@incubator.apache.org Subject: Connector architecture question and suggestion Hi, I am having a look at the connectors. At the moment to my opinion the classes for (all) connectors are to big. This is partly due to the way the interfaces are structured and partly due to the implementation of html in java. For example the RssConnector now has almost 6000 lines of code and the JdbcConnector has almost 2000 lines of code. To my opinion this can be improved by making separate components for presenting and configuring the connectors in the crawler-ui web project and for the part needed by the runner. Abstracting the html from the actual classes can help a lot as well. Maybe some utility methods to make creating these html pages easier is nice as well. I am willing to investigate this path further, but I'd like to have ideas of other developers. It would be nice to know if others feel like this can be improved as well. It might be interesting to look at a technique like wicket for the ui part. Than you can package the html code together with the java code in one jar. No difficult repackaging is required and you can still create nice interfaces. I also read that others want to have a look at something like velocity, of course that can be a valid option as well. So what is your opinions about it? regards Jettro Coenradie http://www.gridshore.nl
Re: Connector architecture question and suggestion
Oke, I will come up with a proposal for breaking up the components in a way that still enables us to easily keep the current adapters in their own structure. As for the UI testen, it is always a beast. We do have some experience with tools like Webdriver and others. Will have a look at that as well. On Mon, Aug 16, 2010 at 2:41 PM, karl.wri...@nokia.com wrote: I think that providing tools/help for implementing the UI pieces of connectors is a perfectly reasonable thing to do. However, I strongly believe that the UI components should remain described as part of the connector interfaces. Breaking their implementations out within individual connectors is also reasonable, but unless there is a compelling reason to refactor all the individual connectors in that way, I would hold off that project until there are better unit tests for the connector UI components. If you are willing to contribute such tests, I think going that way would be worth consideration. I'd like to see a more detailed proposal before I comment further. Thanks, Karl -Original Message- From: jettro.coenra...@gmail.com [mailto:jettro.coenra...@gmail.com] On Behalf Of ext Jettro Coenradie Sent: Monday, August 16, 2010 7:37 AM To: connectors-dev@incubator.apache.org Subject: Connector architecture question and suggestion Hi, I am having a look at the connectors. At the moment to my opinion the classes for (all) connectors are to big. This is partly due to the way the interfaces are structured and partly due to the implementation of html in java. For example the RssConnector now has almost 6000 lines of code and the JdbcConnector has almost 2000 lines of code. To my opinion this can be improved by making separate components for presenting and configuring the connectors in the crawler-ui web project and for the part needed by the runner. Abstracting the html from the actual classes can help a lot as well. Maybe some utility methods to make creating these html pages easier is nice as well. I am willing to investigate this path further, but I'd like to have ideas of other developers. It would be nice to know if others feel like this can be improved as well. It might be interesting to look at a technique like wicket for the ui part. Than you can package the html code together with the java code in one jar. No difficult repackaging is required and you can still create nice interfaces. I also read that others want to have a look at something like velocity, of course that can be a valid option as well. So what is your opinions about it? regards Jettro Coenradie http://www.gridshore.nl -- Jettro Coenradie http://www.gridshore.nl