Hi Alex,
Some great ideas, all of which are deserving of comment.
- There is code commented out, but much of this code seems to still be
valuable, like it was commented from some migrations and was left over for
somebody to follow-up (e.g. unit tests).
True. Some intelligence is required. When in doubt, leave it - but there are a lot of
things that are obviously moved or old rewritten code. This is all volunteer and just
getting people involved with "baby steps" would be great. I would also hope
that some inactive authors come back and clean up comments in their own code. Or write
those unit tests if that was the intention. There are TODO comments in the code that
could be tackled.
- There are issues reported by SonnarQube [1] like:
This should be handled with kid gloves. A lot of those reports cover items
that are not yet complete, ordered for easier following / understanding of
code, etc. However a lot can be handled easily and quickly, like adding
@Override ... People can use local plugins that check code like findBugs. I
used to be religious about it but have become lax. This is a good reminder for
me to start again.
- Removal of hardcoded paths like: "/tmp",
I am in complete agreement. Things like /tmp should probably even be
refactored to use temp files. Things like default paths used in static
createAnnotatorDescription() should instead probably be used in
@ConfigurationParameter default= ...
--- Building upon that statement, it would be nice to migrate older annotation
engines, readers, and cas consumers to the uimafit paradigm. This would help a
newbie understand the difference and how to use AEs, etc.
- Migrate scripts from Ant (files like build-*.xml) to maven.
Does ctakes have these? I guess that I've missed them. Yeah, full maven would
be nice.
- Deprecated code
We certainly have a lot of it. It is a good excuse to make unit tests before
updating.
- I think it is time to define some conventions for:
- formatting (identation),
- crlf conventions (see .gitattributes)
- etc
You are correct; indentation and crlf should also be settable by a decent ide
for any cvs. I think that most ctakes code is space indented, 3 per
indentation, and \n only for newlines. I could be wrong.
Things that are more stylistic (naming, ordering, etc.) are much more coder-
preference. I would rather have contributions than turn people off with
strictures. I'll even take things like missing { } ... though there is
another great target for refactoring ...
- For git vs Subversion, I am able to use the same folder with a .git
Thanks for the documentation! As an Apache project we would need to vote on
fully moving to git (as Tim and Dave suggested). I am definitely not opposed
to that - I use github for everything else these days ...
- There are commits without any reference to Jira issues or other type
Guilty as charged. A lot of my commits are new development and I only write
commit comments. I could open a jira for each, but I am admittedly lazy about
such things. Ditto for placing links in an email appendix.
Also, based on the decision to use semantic versioning, it
will need to choose between 4.0.1 or 4.1.0.
Personally I think that our next release should be 4.1.0 as there are enough
new features to distinguish that it isn't just a patch release.
http://semver.org/
Thanks,
Sean
-----Original Message-----
From: Alexandru Zbarcea [mailto:al...@apache.org]
Sent: Monday, November 20, 2017 8:34 AM
To: Apache cTAKES Dev; Hadrian Zbarcea
Subject: Re: Contribute to ctakes: it is in your best interests! RE: unknown
dependencies [EXTERNAL] [SUSPICIOUS] [SUSPICIOUS]
Hi,
To grow the community and bring even more adoption is my desire, too. I cannot
agree more with what you said, Sean, Tim.
I have discussed with Hadrian (Apache member) about cTAKES adoption and I think
he has great ideas about the priorities for this community to grow. I will like
to introduce him to the community and let him express some ideas.
In regards to the technical issues that where already identified on this
thread, I would like to understand your perspective and prioritization.
- There is code commented out, but much of this code seems to still be
valuable, like it was commented from some migrations and was left over for
somebody to follow-up (e.g. unit tests).
- There are issues reported by SonnarQube [1] like:
- 3.3K bugs [2]
- 16.5% code duplication (24K LoC) [3]
174 bugs in the last month [4]
- I would like to see more Unit Tests for the code. There are new
commits unrelated to a feature description and so, there is no clear
understanding about what the review should focus on. I think it relates to
the same request from Sean to have "sanity-test type unit tests - Little
two or three-line "does this method crack" tests.". I see this task as one
of the most important one.
- Removal of hardcoded paths like: "/tmp",
"C:/Users/<some-user>/<some-path>.
- Migrate scripts from Ant (files like build-*.xml) to maven. It makes
the code so unpredictable. I find it difficult to navigate through these
when tests are dependent upon these executions.
- Classpaths manually specified.
- Deprecated code
- Old libraries which involve security risks in production (e.g. Spring
that was just upgraded)
Other tasks that are related more to productivity.
- I think it is time to define some conventions for:
- formatting (identation),
- crlf conventions (see .gitattributes)
- etc
- For git vs Subversion, I am able to use the same folder with a .git
and .svn VCS and documented on the wiki [5].
- There are commits without any reference to Jira issues or other type
of documentation. In consequence, when release will come, it will be very
hard to hunt those changes and understand why those commits were made: bugs
vs features. Also, based on the decision to use semantic versioning, it
will need to choose between 4.0.1 or 4.1.0.
My $0.02,
Alex
[1] -
https://urldefense.proofpoint.com/v2/url?u=https-3A__builds.apache.org_analysis_overview-3Fid-3Dorg.apache.ctakes-253Actakes&d=DwIBaQ&c=qS4goWBT7poplM69zy_3xhKwEW14JZMSdioCoppxeFU&r=fs67GvlGZstTpyIisCYNYmQCP6r0bcpKGd4f7d4gTao&m=PHstasp4Y8wYPWGquySLw2RPNP-d8XkCTXvOuP-YWuI&s=ZBpW0OVPlYu308dmEv3E6DK93VfUe8NLi0OClLqa2Sk&e=
[2] -
https://urldefense.proofpoint.com/v2/url?u=https-3A__builds.apache.org_analysis_component-5Fissues-3Fid-3Dorg.apache.ctakes-253Actakes-23resolved-3Dfalse-257Ctypes-3DBUG&d=DwIBaQ&c=qS4goWBT7poplM69zy_3xhKwEW14JZMSdioCoppxeFU&r=fs67GvlGZstTpyIisCYNYmQCP6r0bcpKGd4f7d4gTao&m=PHstasp4Y8wYPWGquySLw2RPNP-d8XkCTXvOuP-YWuI&s=Vot25EW4XwGjz9uLwHo4rc62shM_0n-6Yy5u9BjktsM&e=
[3] -
https://urldefense.proofpoint.com/v2/url?u=https-3A__builds.apache.org_analysis_component-5Fmeasures_metric_duplicated-5Fblocks_list-3Fid-3Dorg.apache.ctakes-253Actakes&d=DwIBaQ&c=qS4goWBT7poplM69zy_3xhKwEW14JZMSdioCoppxeFU&r=fs67GvlGZstTpyIisCYNYmQCP6r0bcpKGd4f7d4gTao&m=PHstasp4Y8wYPWGquySLw2RPNP-d8XkCTXvOuP-YWuI&s=NKhS3KX3JBBiuFbfjPSq2WT-qibS-QSQzqkG8KbiLIk&e=
[4] -
https://urldefense.proofpoint.com/v2/url?u=https-3A__builds.apache.org_analysis_component-5Fissues-3Fid-3Dorg.apache.ctakes-253Actakes-23resolved-3Dfalse-257Ctypes-3DBUG-257CsinceLeakPeriod-3Dtrue&d=DwIBaQ&c=qS4goWBT7poplM69zy_3xhKwEW14JZMSdioCoppxeFU&r=fs67GvlGZstTpyIisCYNYmQCP6r0bcpKGd4f7d4gTao&m=PHstasp4Y8wYPWGquySLw2RPNP-d8XkCTXvOuP-YWuI&s=tNsgiXoIKXQPQAzM7g-EEXEephKMNEG50OBl8iuD6lU&e=
[5] -
https://urldefense.proofpoint.com/v2/url?u=https-3A__cwiki.apache.org_confluence_display_CTAKES_cTAKES-2B4.0-2BDeveloper-2BInstall-2BGuide-23cTAKES4.0DeveloperInstallGuide-2DSubversion-2BGit&d=DwIBaQ&c=qS4goWBT7poplM69zy_3xhKwEW14JZMSdioCoppxeFU&r=fs67GvlGZstTpyIisCYNYmQCP6r0bcpKGd4f7d4gTao&m=PHstasp4Y8wYPWGquySLw2RPNP-d8XkCTXvOuP-YWuI&s=lMZ18SEZob73AXp4a3sMrd22nHpwFtQ__4fR-Q5QQuI&e=
On Mon, Nov 20, 2017 at 6:32 AM, Miller, Timothy <
timothy.mil...@childrens.harvard.edu> wrote:
Git is available to apache projects, and many projects have moved over
(see here:
https://urldefense.proofpoint.com/v2/url?u=https-3A__git-2Dwip-2Dus.apache.org_repos_asf&d=DwIBaQ&c=qS4goWBT7poplM69zy_3xhKwEW14JZMSdioCoppxeFU&r=fs67GvlGZstTpyIisCYNYmQCP6r0bcpKGd4f7d4gTao&m=PHstasp4Y8wYPWGquySLw2RPNP-d8XkCTXvOuP-YWuI&s=qGV9tIcYJGK-tQAMYm5cWevWrBSixPCHj3VfaXum288&e=):
Here is the general info on what that looks like:
https://urldefense.proofpoint.com/v2/url?u=https-3A__www.apache.org_de
v_writable-2Dgit&d=DwIBaQ&c=qS4goWBT7poplM69zy_3xhKwEW14JZMSdioCoppxeF
U&r=fs67GvlGZstTpyIisCYNYmQCP6r0bcpKGd4f7d4gTao&m=PHstasp4Y8wYPWGquySL
w2RPNP-d8XkCTXvOuP-YWuI&s=BRSYUV67HZtyxzLNbqPzAlS-YZmqUpA30rvPsNKX6i0&
e=
A few points from that link:
Projects can request moving to Git as their main code repository, by
creating an INFRA issue. See also the infra-contact page. > Projects
can request new, blank repositories by using reporeq.apache.org.
The current system has basic git support only. We are working on
extending this service in the near future.
Custom commit or other hooks will not be supported, all projects get
the
same hooks. Setting up gitpubsub should provide sufficient flexiblity
without impacting the core Git setup, volunteers are welcome to make
that happen.
(Not sure what basic support only means.)
There are also read-only git repos available by default for every
project and updated in near-real-time:
https://urldefense.proofpoint.com/v2/url?u=https-3A__www.apache.org_de
v_git.html&d=DwIBaQ&c=qS4goWBT7poplM69zy_3xhKwEW14JZMSdioCoppxeFU&r=fs
67GvlGZstTpyIisCYNYmQCP6r0bcpKGd4f7d4gTao&m=PHstasp4Y8wYPWGquySLw2RPNP
-d8XkCTXvOuP-YWuI&s=CtgGvLG2s_KqVRWx_tZAcaMSh_KKH4aqc6HGTP3dmtA&e=
with those I guess the suggested workflow is to work off of that repo
and then just submit patches to someone who commits with svn rather
than committing directly.
I've been using the git-svn connector myself recently since I just
vastly prefer the git lightweight branching for focused development,
as it helps me keep a cleaner working directory. But that adds some
additional annoying steps.
Tim
________________________________________
From: Finan, Sean <sean.fi...@childrens.harvard.edu>
Sent: Saturday, November 18, 2017 1:23 PM
To: dev@ctakes.apache.org
Subject: RE: Contribute to ctakes: it is in your best interests! RE:
unknown dependencies [EXTERNAL] [SUSPICIOUS] [SUSPICIOUS]
Hi Dave,
Those are some great thoughts. Being an apache project I am not sure
how far we can move from svn, but there may be a way. You are not the
first to voice this desire for an active github repo and I'm sure that
you won't be the last.
I completely agree with your discussion board preference. Do you have
any recommendations?
You make a great point regarding documentation. In reference to
things that anybody can quickly contribute ... that would be a big one.
Volunteers?!?
I am really happy to hear that you want to contribute - more than you
already have, which is actually quite a bit!
Cheers,
Sean
-----Original Message-----
From: David Kincaid [mailto:kincaid.d...@gmail.com]
Sent: Saturday, November 18, 2017 1:10 PM
To: dev@ctakes.apache.org
Subject: Re: Contribute to ctakes: it is in your best interests! RE:
unknown dependencies [EXTERNAL] [SUSPICIOUS]
Sean, I can share a couple things that have been an obstacle for me.
It may seem a minor point to some, but I left Subversion behind years
ago and really have no desire to go back. If the project were moved
over to Git/Github it would really smooth the way for me at least. I
would be happy to help out with this. One of the other things I would
really like to see is the mailing list moved onto a discussion board
platform. It seems to me that a discussion board style of tool tends
to create a more active community than a mailing list does.
The other thing that might help get new people involved is making it
easier to find information about the development environment. Things
like branching strategies, coding conventions, etc are really hard to
find from the main cTAKES web site. I saw some references to Jenkins
builds recently on the list. I had no idea there was a Jenkins CI
server for the project somewhere. It also takes some digging to find a
link to Jira. Maybe we could create a Wiki page that describes where
all these tools are and how they are used.
You guys have really done some great work over the last couple of
years cleaning up the code base and improving the documentation by a
ton. Things like the fast dictionary annotator, dictionary creator GUI
are a great addition and make it a lot easier for other people to get
up and running more quickly. As I'm ramping up my research as well as
some proof of concept stuff at work I'll be working more and more with
cTAKES and would love to contribute more to the project.
Just my thoughts.
- Dave
On Sat, Nov 18, 2017 at 11:10 AM, Finan, Sean <
sean.fi...@childrens.harvard.edu> wrote:
Hi Tim, Alex,
Great ideas. I like your (Tim) idea to 1. start with commented code
removal.
Then maybe move on to
2. sanity-test type unit tests - Little two or three-line "does this
method crack" tests.
And another that is simply
3. "populate a test cas with type(s) X" and a factory with
"getSectionTestCas" "getSetenceTestCas" "getPosTestCas" "getChunkTestCas"
... just really simple reusables for tests.
Then
4. refactor to extract and consolidate duplicate code - it is all
over the place ...
These are just my initial thoughts and suggestions, but I think that
those
4 tasks can be performed by anybody of any experience level. They build
upon each other and should help the implementers better understand
ctakes.
After that the sky is the limit.
A couple of years ago I sat on a panel at a workshop for open source
scientific software. For the half dozen or so highlighted projects
(ctakes was one!) the common thread was that getting people to
contribute is extremely difficult.
I have a tendency to assume that people always act in their best
interests. Any student thinking of going towards industry should be
jumping at the opportunity to contribution to a large,
production-quality project. They should also realize that
contribution means potential recommendation (and possibly hiring
interest) by established developers, physicians and researchers that
use ctakes. Even just answering questions on a user or dev list
creates
credibility and can build a network.
Active researchers could discover common thoughts and directions
that could lead to collaboration outside ctakes. Researchers and
companies trying to build upon open source should realize that
direct contribution is easier than custom substitution. Plus, it is
in their best interests that code does what they need it to do in
the fastest, lightest, most stable way possible.
With a project like ctakes there are a lot of things that can be
done, there are great opportunities to really shine. "I wrote this
tool for my thesis that performs some nlp task" sounds good.
Appending "in an Apache product and it has been taken up by thousands across the
globe"
makes it sound a lot better.
At my previous job in industry the company actively contributed to
several open source projects. We had a few people for whom that was
50% of their job. Why? Because we made a commitment to use that
open
source software.
It was a better use of our resources to contribute to it, improve it
and keep its momentum going and prevent it from becoming stale (or
abandoned) while our software continued to move forward.
Hmm, that was a touch more than I had planned to write. A whole cup
of coffee in that one.
Sean
-----Original Message-----
From: Miller, Timothy [mailto:timothy.mil...@childrens.harvard.edu]
Sent: Saturday, November 18, 2017 8:13 AM
To: dev@ctakes.apache.org
Subject: Re: unknown dependencies [EXTERNAL] [SUSPICIOUS]
Thanks Alex, looks like that was probably a fat-fingered auto-import
on my part.
I like your idea, and I don't know the best way to to start either,
but maybe one suggestion is to start with one or two focused things
to clean up, and then ask for volunteers to take on specific modules?
Then people can contribute an hour here and there to do cleanup on
their task/module and try to fix that thing in a 1-2-month long
sprint. I am happy to contribute to cleanup, I am responsible for my
fair share of unclean code, but since I don't have strong software
engineering chops it would be good to have people with that
background propose the tasks and describe exactly what needs to be
done. My idea of cleaning is just to delete commented out sections of
evaluation code.
Tim
________________________________________
From: Alexandru Zbarcea <al...@apache.org>
Sent: Friday, November 17, 2017 4:46 PM
To: Apache cTAKES Dev
Subject: unknown dependencies [EXTERNAL]
Hi,
I notice that a miss-dependency has slipped in the code:
jdk.internal.org.objectweb.asm.commons.AnalyzerAdapter;
Now, that the Jenkins builds is successful, I think it is easier to
clean-up the code. I would like to be a common effort. I don't know
the best way to approach this.
Looking forward to your advice,
Alex
