[GitHub] nifi issue #2671: NiFi-5102 - Adding Processors for MarkLogic DB

2018-05-14 Thread vivekmuniyandi 
Github user vivekmuniyandi commented on the issue:

https://github.com/apache/nifi/pull/2671
  
@joewitt I have addressed all your comments except for the License and 
Notice comments. Can you please let us know what more should we add apart from 
the LICENSE and NOTICE file prepared by our legal team which we have included 
in the root directory of the nar? That constitutes for all the dependecies 
added. What more should be added? Please help. Thanks.


---

[GitHub] nifi issue #2671: NiFi-5102 - Adding Processors for MarkLogic DB

2018-05-14 Thread vivekmuniyandi 
Github user vivekmuniyandi commented on the issue:

https://github.com/apache/nifi/pull/2671
  
@MikeThomsen Sure, will add that to our backlog. Thanks!


---

[GitHub] nifi issue #2671: NiFi-5102 - Adding Processors for MarkLogic DB

2018-05-10 Thread vivekmuniyandi 
Github user vivekmuniyandi commented on the issue:

https://github.com/apache/nifi/pull/2671
  
Thanks @joewitt for the comment. I am addressing all the changes you have 
mentioned. I will address the SSLContext and remove the Kerberos and 
Certificate auth for now. 

``` The other thing that needs to happen is the nar bundles need their 
LICENSE/NOTICE file(s) added if necessary. I looked at one of the nars and 
there would definitely need to be entries.```

Wrt this, I have a 
[LICENSE](https://github.com/apache/nifi/pull/2671/files#diff-53deed39bf31085fbecf77ea6a2382dc)
 and 
[NOTICE](https://github.com/apache/nifi/pull/2671/files#diff-a2f6b487a7a70d5f43fa320730b2c87a)
 file prepared by our legal team in the root directory (to account for the 
contents of the root directory and the sub directories) of the nifi marklogic 
bundle. That constitutes for all the dependecies added in our MarkLogic bundle. 
Should we do something more? Can you explain a bit more here as to what is 
required? 

Thanks for all the help?


---

[GitHub] nifi issue #2671: NiFi-5102 - Adding Processors for MarkLogic DB

2018-05-07 Thread vivekmuniyandi 
Github user vivekmuniyandi commented on the issue:

https://github.com/apache/nifi/pull/2671
  
@MikeThomsen We don't have a public MarkLogic Docker image but we do have 
[this](https://hub.docker.com/r/patrickmcelwee/marklogic-dependencies/) on 
Docker Hub which would give you a head start on having a MarkLogic instance up 
and running. 

I will drop them an email and I will work on getting a secure access to the 
cluster. Thanks for all the help. 


---

[GitHub] nifi issue #2671: NiFi-5102 - Adding Processors for MarkLogic DB

2018-05-04 Thread vivekmuniyandi 
Github user vivekmuniyandi commented on the issue:

https://github.com/apache/nifi/pull/2671
  
Thanks @MikeThomsen for the comment. `PutMarkLogicRecord` is definitely on 
our roadmap but I am not sure when we will be able to get to it. We have an 
internal sprint for NiFi. We will add this to our backlog, check with PM and 
address this with priority. 

We don't want to keep this PR waiting for that processor. I would raise a 
separate PR in the future for that. Thanks!


---

[GitHub] nifi issue #2671: NiFi-5102 - Adding Processors for MarkLogic DB

2018-05-03 Thread vivekmuniyandi 
Github user vivekmuniyandi commented on the issue:

https://github.com/apache/nifi/pull/2671
  
Thanks @MikeThomsen ! Have made the changes. 


---

[GitHub] nifi pull request #2671: NiFi-5102 - Adding Processors for MarkLogic DB

2018-05-03 Thread vivekmuniyandi 
Github user vivekmuniyandi commented on a diff in the pull request:

https://github.com/apache/nifi/pull/2671#discussion_r185979858
  
--- Diff: 
nifi-nar-bundles/nifi-marklogic-bundle/nifi-marklogic-processors/src/main/java/com/marklogic/nifi/processor/QueryMarkLogic.java
 ---
@@ -0,0 +1,145 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements.  See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License.  You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package com.marklogic.nifi.processor;
+
+import com.marklogic.client.datamovement.ExportListener;
+import com.marklogic.client.ext.datamovement.job.SimpleQueryBatcherJob;
+import com.marklogic.client.io.BytesHandle;
+import org.apache.nifi.annotation.documentation.CapabilityDescription;
+import org.apache.nifi.annotation.documentation.Tags;
+import org.apache.nifi.components.PropertyDescriptor;
+import org.apache.nifi.flowfile.FlowFile;
+import org.apache.nifi.processor.ProcessContext;
+import org.apache.nifi.processor.ProcessSession;
+import org.apache.nifi.processor.ProcessSessionFactory;
+import org.apache.nifi.processor.ProcessorInitializationContext;
+import org.apache.nifi.processor.Relationship;
+import org.apache.nifi.processor.exception.ProcessException;
+import org.apache.nifi.processor.util.StandardValidators;
+
+import java.util.ArrayList;
+import java.util.Collections;
+import java.util.HashSet;
+import java.util.List;
+import java.util.Set;
+
+
+@Tags({"MarkLogic"})
+@CapabilityDescription("Creates FlowFiles from batches of documents, 
matching the given criteria," +
+" retrieved from a MarkLogic server using the MarkLogic Data Movement 
SDK (DMSDK)")
+public class QueryMarkLogic extends AbstractMarkLogicProcessor {
--- End diff --

We have added this as a task in our sprint and accommodate in the coming 
weeks. 


---

[GitHub] nifi issue #2671: NiFi-5102 - Adding Processors for MarkLogic DB

2018-05-03 Thread vivekmuniyandi 
Github user vivekmuniyandi commented on the issue:

https://github.com/apache/nifi/pull/2671
  
I followed this link - 
https://cwiki.apache.org/confluence/display/NIFI/Contributor+Guide#ContributorGuide-Keepingyourfeaturebranchcurrent

and looks like this pulled other's commits as well.


---

[GitHub] nifi pull request #2671: NiFi-5102 - Adding Processors for MarkLogic DB

2018-05-02 Thread vivekmuniyandi 
GitHub user vivekmuniyandi opened a pull request:

https://github.com/apache/nifi/pull/2671

NiFi-5102 - Adding Processors for MarkLogic DB

…tMarkLogic

Thank you for submitting a contribution to Apache NiFi.

In order to streamline the review of the contribution we ask you
to ensure the following steps have been taken:

### For all changes:
- [x] Is there a JIRA ticket associated with this PR? Is it referenced 
 in the commit message?
Yes, there is a JIRA ticket - 
https://issues.apache.org/jira/browse/NIFI-5102

- [x] Does your PR title start with NIFI- where  is the JIRA number 
you are trying to resolve? Pay particular attention to the hyphen "-" character.
Yes

- [x] Has your PR been rebased against the latest commit within the target 
branch (typically master)?
Yes

- [x] Is your initial contribution a single, squashed commit?
No

### For code changes:
- [x] Have you ensured that the full suite of tests is executed via mvn 
-Pcontrib-check clean install at the root nifi folder?
Yes, but certain tests from other processors fail - such as from grpc 
processors were failing with an unapproved license error for auto-generated 
files in the target folder.

- [x] Have you written or updated unit tests to verify your changes?
Yes

- [x] If adding new dependencies to the code, are these dependencies 
licensed in a way that is compatible for inclusion under [ASF 
2.0](http://www.apache.org/legal/resolved.html#category-a)? 
Yes

- [x] If applicable, have you updated the LICENSE file, including the main 
LICENSE file under nifi-assembly?
No
- [x] If applicable, have you updated the NOTICE file, including the main 
NOTICE file found under nifi-assembly?
Yes

Have added a NOTICE file and a LICENSE file under the nifi-marklogic-bundle 
project

- [x] If adding new Properties, have you added .displayName in addition to 
.name (programmatic access) for each of the new properties?
Yes

### For documentation related changes:
- [x] Have you ensured that format looks appropriate for the output in 
which it is rendered?
Yes

### Note:
Please ensure that once the PR is submitted, you check travis-ci for build 
issues and submit an update to your PR as soon as possible.


You can merge this pull request into a Git repository by running:

$ git pull https://github.com/marklogic/nifi nifi-5102

Alternatively you can review and apply these changes as the patch at:

https://github.com/apache/nifi/pull/2671.patch

To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:

This closes #2671


commit 16f8ffef0e0a853535c5af63b7e28441b4674c47
Author: Vivek Siddharthan Muniyandi 
Date:   2018-05-03T03:01:25Z

NIFI-5102 Adding MarkLogic DB NiFi processors - QueryMarkLogic and 
PutMarkLogic

commit e7b4bee69b6bd6a5274613ff884625a9d30b5ffb
Author: Vivek Siddharthan Muniyandi 
Date:   2018-05-03T03:26:14Z

NIFI-5102 Added displaynames for Properties




---