[GitHub] nifi issue #2028: NIFI-3518 Create a Morphlines processor

[WilliamNouet]

Github user WilliamNouet commented on the issue:

https://github.com/apache/nifi/pull/2028
  
@joewitt & @binhnv would you have an update on the above recent comments 
and modifications?
Thanks!


---

[GitHub] nifi issue #2028: NIFI-3518 Create a Morphlines processor

[WilliamNouet]

Github user WilliamNouet commented on the issue:

https://github.com/apache/nifi/pull/2028
  
@binhnv I could move it to the nifi-kite-bundle if needed, but even if they 
both leverages the kite dependencies, they have different outcomes: one regards 
limited data conversions (avro, json, csv) and storage, whereas the other one 
offer way more flexibly in terms of data transformations (please refer to the 
Morphlines doc), and as such should be a subset of its own as it could be 
developed further in the future. As such, it makes sense for me to keep it 
separate.


---

[GitHub] nifi issue #2028: NIFI-3518 Create a Morphlines processor

[binhnv]

Github user binhnv commented on the issue:

https://github.com/apache/nifi/pull/2028
  
@WilliamNouet there is already a `nifi-kite-bundle` which has `kite`'s 
dependencies, does it make sense to move this processor to that bundle?


---

[GitHub] nifi issue #2028: NIFI-3518 Create a Morphlines processor

[WilliamNouet]

Github user WilliamNouet commented on the issue:

https://github.com/apache/nifi/pull/2028
  
@joewitt thanks for the feedback! That was quite a bit of changes to make 
on my side, but hopefully the latest commit captures everything. I will need to 
work on the rebase a bit later though.


---

[GitHub] nifi issue #2028: NIFI-3518 Create a Morphlines processor

[joewitt]

Github user joewitt commented on the issue:

https://github.com/apache/nifi/pull/2028
  
i added a file to the JIRA which is a patch you can apply for the simple 
pom updates for version


---

[GitHub] nifi issue #2028: NIFI-3518 Create a Morphlines processor

[joewitt]

Github user joewitt commented on the issue:

https://github.com/apache/nifi/pull/2028
  
Bill,

Some feedback after reviewing in more detail.

1. There are no unit tests.  I can see why they'd be non-portable given it 
is executing a process but it does seem like a good candidate for an 
integration-test at least.  We should definitely have something.  We do have a 
naming convention for integration tests to execute.
2. The nar is not connected to the nifi-nar-bundles pom
3. The nar is not connected to the nifi-assembly.  This is perfectly fine 
though if you dont want the nar to be available in the build by default but 
rather something someone opts into.  I'm ok either way just pointing this out.
4. The L updates within the nar are a good start but are incomplete with 
regard to the totality of the jars present within the nar.
5. The L updates in the nar will also need to be validated to see if any 
need to be carried forward into the nifi-assembly nar.
6. There are unused imports which cause checkstyle/rat failures which is 
why travis failed on all builds.
7. I'm not an expert on morphlines but it seems like it could be used to do 
dangerous things on the filesystem.  We have a RestrictedComponent annotation 
for this purpose so it is probably best to add that to the processor.
8. The processor name for a 'users point of view' should probably be 
'ExecuteMorphline'
9. You'll probably want to provide some documentation/additional 
documentation as many other processors do helping the user (and reviewer to 
evaluate).
10. Can you update to latest nifi version which is 1.6.0-SNAPSHOT in your 
next commit.  Not a big deal as I can do that easily enough but it helps.

This looks like a lot but for the L we have tons of examples.  The rest 
are pretty straightforward to sort out.

Let me know if you plan to tackle these.

Thanks



---

[GitHub] nifi issue #2028: NIFI-3518 Create a Morphlines processor

[WilliamNouet]

Github user WilliamNouet commented on the issue:

https://github.com/apache/nifi/pull/2028
  
@joewitt, did you get a chance to review the latest commit?


---

[GitHub] nifi issue #2028: NIFI-3518 Create a Morphlines processor

[WilliamNouet]

Github user WilliamNouet commented on the issue:

https://github.com/apache/nifi/pull/2028
  
@joewitt Thanks for getting back to me, and sorry for the delay; I was away 
without access to a laptop. I just pushed a commit with an updated snapshot 
version and licensing.


---

[GitHub] nifi issue #2028: NIFI-3518 Create a Morphlines processor

[joewitt]

Github user joewitt commented on the issue:

https://github.com/apache/nifi/pull/2028
  
@WilliamNouet  can you please see my last set of comments.  A rebase and 
some progress on the licensing will be necessary. I can definitely help that.  
I have some time over the next couple of days and this is one I'd be happy to 
try and knock out if time allows.


---

[GitHub] nifi issue #2028: NIFI-3518 Create a Morphlines processor

[joewitt]

Github user joewitt commented on the issue:

https://github.com/apache/nifi/pull/2028
  
@WilliamNouet Sorry for the trouble.  We try to have a good review model 
for these things and it seems like we just dont have anyone familiar with 
morphlines that can help.  I am happy to help review it and work with you to 
get it in though.  Can you please rebase, change nifi version references to 
1.5.0-SNAPSHOT, and we can go from there.  I'm almost certain there will be a 
lot of LICENSE and NOTICE updates required which I dont see yet.


---

[GitHub] nifi issue #2028: NIFI-3518 Create a Morphlines processor

[WilliamNouet]

Github user WilliamNouet commented on the issue:

https://github.com/apache/nifi/pull/2028
  
Dear all,

First, let me introduce myself. My name is William Nouet and I am a big fan 
of the NiFi project. I believe this is revolutionizing the way the industry is 
handling data transport and transformation. I would love to be involved in the 
project and its open source community; so I created a JIRA (NIFI-3518) and 
decided to help close another one (NIFI-385).

I got some amazing feedback and guidance from the community while working 
on the above 2 JIRAs, updated my code to make sure it lives up to the NiFi 
standards. That being said, I haven’t been able to get any traction on 
getting the code pushed to the main branch for the past few months, hence this 
emailZ

Please let me know if there is anything I can do to get the ball rolling. I 
would be happy to re-work on my code if you think it can be improved and would 
more than happy to meet in person in a bar or a coffee shop if some of you guys 
are in the NYC metro area.

Thank you in advance.

Best,

William

Envoyé de mon iPhone

> Le 6 oct. 2017 à 16:16, Joe Witt  a écrit :
> 
> Is there anyone familiar with morphlines that can help test/review?
> 
> —
> You are receiving this because you authored the thread.
> Reply to this email directly, view it on GitHub, or mute the thread.
> 



---

[GitHub] nifi issue #2028: NIFI-3518 Create a Morphlines processor

[joewitt]

Github user joewitt commented on the issue:

https://github.com/apache/nifi/pull/2028
  
Is there anyone familiar with morphlines that can help test/review?


---