Connector architecture question and suggestion

2010-08-16 Thread Jettro Coenradie
Hi,
I am having a look at the connectors. At the moment to my opinion the
classes for (all) connectors are to big. This is partly due to the way the
interfaces are structured and partly due to the implementation of html in
java. For example the RssConnector now has almost 6000 lines of code and the
JdbcConnector has almost 2000 lines of code. To my opinion this can be
improved by making separate components for presenting and configuring the
connectors in the crawler-ui web project and for the part needed by the
runner. Abstracting the html from the actual classes can help a lot as well.
Maybe some utility methods to make creating these html pages easier is nice
as well.

I am willing to investigate this path further, but I'd like to have ideas of
other developers. It would be nice to know if others feel like this can be
improved as well.

It might be interesting to look at a technique like wicket for the ui part.
Than you can package the html code together with the java code in one jar.
No difficult repackaging is required and you can still create nice
interfaces. I also read that others want to have a look at something like
velocity, of course that can be a valid option as well.

So what is your opinions about it?

regards

Jettro Coenradie
http://www.gridshore.nl


RE: Connector architecture question and suggestion

2010-08-16 Thread karl.wright
I think that providing tools/help for implementing the UI pieces of connectors 
is a perfectly reasonable thing to do.  However, I strongly believe that the UI 
components should remain described as part of the connector interfaces.  
Breaking their implementations out within individual connectors is also 
reasonable, but unless there is a compelling reason to refactor all the 
individual connectors in that way, I would hold off that project until there 
are better unit tests for the connector UI components. If you are willing to 
contribute such tests, I think going that way would be worth consideration.

I'd like to see a more detailed proposal before I comment further.

Thanks,
Karl

-Original Message-
From: jettro.coenra...@gmail.com [mailto:jettro.coenra...@gmail.com] On Behalf 
Of ext Jettro Coenradie
Sent: Monday, August 16, 2010 7:37 AM
To: connectors-dev@incubator.apache.org
Subject: Connector architecture question and suggestion

Hi,
I am having a look at the connectors. At the moment to my opinion the
classes for (all) connectors are to big. This is partly due to the way the
interfaces are structured and partly due to the implementation of html in
java. For example the RssConnector now has almost 6000 lines of code and the
JdbcConnector has almost 2000 lines of code. To my opinion this can be
improved by making separate components for presenting and configuring the
connectors in the crawler-ui web project and for the part needed by the
runner. Abstracting the html from the actual classes can help a lot as well.
Maybe some utility methods to make creating these html pages easier is nice
as well.

I am willing to investigate this path further, but I'd like to have ideas of
other developers. It would be nice to know if others feel like this can be
improved as well.

It might be interesting to look at a technique like wicket for the ui part.
Than you can package the html code together with the java code in one jar.
No difficult repackaging is required and you can still create nice
interfaces. I also read that others want to have a look at something like
velocity, of course that can be a valid option as well.

So what is your opinions about it?

regards

Jettro Coenradie
http://www.gridshore.nl


Re: Connector architecture question and suggestion

2010-08-16 Thread Jettro Coenradie
Oke, I will come up with a proposal for breaking up the components in a way
that still enables us to easily keep the current adapters in their own
structure.

As for the UI testen, it is always a beast. We do have some experience with
tools like Webdriver and others. Will have a look at that as well.

On Mon, Aug 16, 2010 at 2:41 PM, karl.wri...@nokia.com wrote:

 I think that providing tools/help for implementing the UI pieces of
 connectors is a perfectly reasonable thing to do.  However, I strongly
 believe that the UI components should remain described as part of the
 connector interfaces.  Breaking their implementations out within individual
 connectors is also reasonable, but unless there is a compelling reason to
 refactor all the individual connectors in that way, I would hold off that
 project until there are better unit tests for the connector UI components.
 If you are willing to contribute such tests, I think going that way would be
 worth consideration.

 I'd like to see a more detailed proposal before I comment further.

 Thanks,
 Karl

 -Original Message-
 From: jettro.coenra...@gmail.com [mailto:jettro.coenra...@gmail.com] On
 Behalf Of ext Jettro Coenradie
 Sent: Monday, August 16, 2010 7:37 AM
 To: connectors-dev@incubator.apache.org
 Subject: Connector architecture question and suggestion

 Hi,
 I am having a look at the connectors. At the moment to my opinion the
 classes for (all) connectors are to big. This is partly due to the way the
 interfaces are structured and partly due to the implementation of html in
 java. For example the RssConnector now has almost 6000 lines of code and
 the
 JdbcConnector has almost 2000 lines of code. To my opinion this can be
 improved by making separate components for presenting and configuring the
 connectors in the crawler-ui web project and for the part needed by the
 runner. Abstracting the html from the actual classes can help a lot as
 well.
 Maybe some utility methods to make creating these html pages easier is nice
 as well.

 I am willing to investigate this path further, but I'd like to have ideas
 of
 other developers. It would be nice to know if others feel like this can be
 improved as well.

 It might be interesting to look at a technique like wicket for the ui part.
 Than you can package the html code together with the java code in one jar.
 No difficult repackaging is required and you can still create nice
 interfaces. I also read that others want to have a look at something like
 velocity, of course that can be a valid option as well.

 So what is your opinions about it?

 regards

 Jettro Coenradie
 http://www.gridshore.nl




-- 
Jettro Coenradie
http://www.gridshore.nl