date:20170329

[GitHub] storm issue #2026: Eventhub3

2017-03-29 Thread rban1

Github user rban1 commented on the issue:

https://github.com/apache/storm/pull/2026
  
@SreeramGarlapati  Made all the relavant changes(Serialization, exception 
handling, enqueuetimefilter)
@harshach I will be using this PR going forward. I have addressed your 
comments on syntax as well. On your comment of backporting the changes to 1.x 
branches it cannot be done as the latest eventhubclient is valid only from java 
8 environment. 


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[GitHub] storm pull request #2026: Eventhub3

2017-03-29 Thread rban1

Github user rban1 commented on a diff in the pull request:

https://github.com/apache/storm/pull/2026#discussion_r108799438
  
--- Diff: 
external/storm-eventhubs/src/main/java/org/apache/storm/eventhubs/spout/EventHubReceiverImpl.java
 ---
@@ -65,77 +57,81 @@ public EventHubReceiverImpl(EventHubSpoutConfig config, 
String partitionId) {
   }
 
   @Override
-  public void open(IEventHubFilter filter) throws EventHubException {
-logger.info("creating eventhub receiver: partitionId=" + partitionId + 
-   ", filterString=" + filter.getFilterString());
+  public void open(String offset) throws EventHubException {
+logger.info("creating eventhub receiver: partitionId=" + partitionId +
+", offset=" + offset);
 long start = System.currentTimeMillis();
-receiver = new ResilientEventHubReceiver(connectionString, entityName,
-   partitionId, consumerGroupName, defaultCredits, filter);
-receiver.initialize();
-
+try {
+  ehClient = 
EventHubClient.createFromConnectionStringSync(connectionString);
+  receiver = ehClient.createEpochReceiverSync(
+  consumerGroupName,
+  partitionId,
+  offset,
+  false,
+  1);
+}catch (Exception e){
+  logger.info("Exception in creating EventhubClient"+e.toString());
+}
 long end = System.currentTimeMillis();
 logger.info("created eventhub receiver, time taken(ms): " + 
(end-start));
   }
 
   @Override
-  public void close() {
+  public void close(){
 if(receiver != null) {
-  receiver.close();
+  try {
+receiver.close().whenComplete((voidargs,error)->{
+  try{
+if(error!=null){
+  logger.error("Exception during receiver close 
phase"+error.toString());
+}
+ehClient.closeSync();
+  }catch (Exception e){
+logger.error("Exception during ehclient close 
phase"+e.toString());
+  }
+}).get();
+  }catch (InterruptedException e){
+logger.error("Exception occured during close phase"+e.toString());
+  }catch (ExecutionException e){
+logger.error("Exception occured during close phase"+e.toString());
+  }
   logger.info("closed eventhub receiver: partitionId=" + partitionId );
   receiver = null;
+  ehClient =  null;
 }
   }
-  
+
+
   @Override
   public boolean isOpen() {
 return (receiver != null);
   }
 
   @Override
-  public EventData receive(long timeoutInMilliseconds) {
+  public EventDataWrap receive() {
 long start = System.currentTimeMillis();
-Message message = receiver.receive(timeoutInMilliseconds);
+Iterable receivedEvents=null;
+/*Get one message at a time for backward compatibility behaviour*/
+try {
+  receivedEvents = receiver.receiveSync(1);
--- End diff --

Made the change


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

Re: [DISCUSS] Ideas for resolving storm-drpc-server compilation issue on IDE

2017-03-29 Thread Bobby Evans

I am fine with those changes so long as we finish the separation of worker and 
daemon classpaths.  Otherwise we have made some very big changes for our end 
users that are going to have a hard time upgrading.
If all we support is the option to run an old worker version with a new 
supervisor/nimbus I think that would be good enough, although I would like to 
see a full separation of the classpaths. 


- Bobby

On Tuesday, March 28, 2017, 6:03:26 PM CDT, Jungtaek Lim  
wrote:Just FYI:
I've worked with minimal patch for 3, though I still don't like such
workaround:
https://github.com/HeartSaVioR/storm/commit/d3122faa7ae182915242b979beaac156f91fe3b2

It excludes 'libthrift', 'jetty', 'codahale metrics' from relocation
targets. I can see IDEA is OK to build the project, and Maven build passing.

- Jungtaek Lim (HeartSaVioR)

2017년 3월 29일 (수) 오전 11:02, Jungtaek Lim 님이 작성:

> Back to origin issue (before breaking down 'storm-core'), turned out
> IntelliJ doesn't recognize relocated classes within project. That's why
> build (via Maven) for master branch succeeds but IDEA compile doesn't.
>
> There're some issues filed but no action has been made.
> https://youtrack.jetbrains.com/issue/IDEA-93855
> https://youtrack.jetbrains.com/issue/IDEA-126596
>
> So suppose we have two modules A and B within project, and A relocates L
> to Lr.
> B relies on A's method which returns a class of Lr or has parameters for a
> class of LR, B needs to use Lr rather than L, and Lr is not recognized from
> B.
>
> Moving 'storm-drpc-server' to 'storm-core' may help but it's not a nice
> solution though. (think about why we add new module 'storm-drpc-server')
> To minimize dependency for worker (which actually affects end users) we
> should break down 'storm-core' and it will remain to be headache.
>
> There seemed to be little workarounds.
>
> 1. Guide IDEA users to take hacky workaround.
>
> Quoting https://youtrack.jetbrains.com/issue/IDEA-93855#comment=27-1838157
> :
> "A hacky workaround is to make the module in intellij with the dependency
> depend on the jar explicitly in target/. This at least allows things to
> compile and tests to run."
>
> That is really bad and annoying, but we might have no choice when we don't
> want to take other workarounds.
>
> 2. Maintaining separate project for relocated dependencies.
>
> This avoids contributors to take hacky workaround so good to go, but
> maintaining relocated artifacts might be another headache, and I'm not sure
> ASF (or LICENSE of relocated targets) allows to do that.
>
> 3. Minimize (or remove) relocate targets and/or don't relocate troublesome
> targets.
>
> For 'storm-drpc-server', there seems to be three troublesome targets:
>
> - 'thrift'
> - 'codahale metrics'
> - 'jetty server' (We may be able to move this to 'storm-drpc-server' when
> another webapp port is done.)
>
> If we are OK to give up relocating those things we might be OK for now. We
> may want to extend the list when we break down more modules from
> 'storm-core'.
>
> Btw, IMHO relocating is not a good option. Elastic gives up shading
> anything for 2.0. (https://www.elastic.co/blog/to-shade-or-not-to-shade)
> Someone might feel that it's a regression, but we need to decide to do it
> when it can provide better shape.
>
> Please add ideas if you have any, and give your opinions about above
> options.
>
> Thanks,
> Jungtaek Lim (HeartSaVioR)
>
> 2017년 3월 28일 (화) 오후 10:23, Bobby Evans 님이 작성:
>
> Sure I am happy to help out how I can.  I really would like to spend more
> time on storm, but sadly work has shifted and my team got 2 new projects
> recently, but we have not increased the head count to cover it yet, so I am
> swamped.  But if you do need help with some of these let me know and I'll
> see what I can do in my spare time.
>
>
> - Bobby
>
> On Tuesday, March 28, 2017, 2:10:46 AM CDT, Jungtaek Lim <
> kabh...@gmail.com> wrote:Bobby,
>
> I just tried to follow your suggestion and found it's less error-prone
> compared to my approach, and has lots of benefits. (I am seeing the great
> chance to minimize dependencies for 'storm-client', say, Worker.)
>
> Thanks for the suggestion. I'm working on this now. I'll mention you when I
> finish working this, or need your help.
>
> Thanks,
> Jungtaek Lim (HeartSaVioR)
>
> 2017년 3월 28일 (화) 오전 8:15, Jungtaek Lim 님이 작성:
>
> I think we could also fix this issue for separating 'storm-core' and
> 'storm-webapp' (rename from 'storm-drpc-server'), since local cluster
> doesn't need to have 'storm-webapp', DRPC server (local DRPC will still be
> in 'storm-core'), UI, Logviewer. That's what I'm working on, which seems to
> require heavy efforts.
>
> Your plan looks really promising, but in other perspective this plan is
> even much harder to address.
> Do you have time frame for working on this? If you can finish the work in
> time frame so that it can be included in 2.0.0, I'll just discard my work
> and move forward to port other things (logviewer, ui) first.
>
> Regarding lo

Re: [RESULT] [VOTE] Release Apache Storm 1.1.0 (RC3)

2017-03-29 Thread Julien Nioche

Hi Jungtaek,

That's strange. It shows as sent but I can't see it in the archives indeed.
Here is what I had reported

*-1*

*I've just compiled 1.1.0 and found that it has the same issue I found
yesterday with 1.0.3 https://issues.apache.org/jira/browse/STORM-2428
.  I could, of course,
rely on an older version of Flux but would rather keep it in sync with
storm-core.*

Nevermind, we'll fix it in 1.1.1! Wasn't a blocker but something good to
fix while the RC was being reviewed.

Thanks

Julien


On 29 March 2017 at 09:48, Jungtaek Lim  wrote:

> Julien,
>
> I can't see your post in RC3 thread, both my inbox and also mail archive.
>
> http://mail-archives.apache.org/mod_mbox/storm-dev/201703.
> mbox/%3C0F8C6435-28BA-4D89-9818-C438575EAC1E%40apache.org%3E
>
> Recently I experienced posting to storm-dev@ sometimes fails 'silently'.
> After that I check mail archive to see it posted after sending mail.
>
> Anyway, could you paste your vote mail again so that we can see why you
> cast -1?
>
> Thanks,
> Jungtaek Lim (HeartSaVioR)
>
> 2017년 3월 29일 (수) 오후 5:20, Julien Nioche 님이
> 작성:
>
> Hi,
>
> For the record and not that it changes the outcome of the vote but I had
> expressed a -1 on the 22th March.
>
> Julien
>
> On 28 March 2017 at 22:09, P. Taylor Goetz  wrote:
>
> > This vote is now closed and passes with X binding +1 votes, X non-binding
> > +1 votes, and no -1 or 0 votes.
> >
> > Vote tally (* indicates a binding vote):
> >
> > +1
> > Jungtaek Lim*
> > Arun Mahadevan*
> > Bobby Evans*
> > Harsha Chintalapani*
> > Hugo Da Cruz Louro*
> > Satish Duggana*
> > Alexandre Vermeerbergen
> > P. Taylor Goetz*
> >
> > I will release the artifacts, allow 24+ hours for mirrors to replicate,
> > and announce the release.
> >
> > Thanks to all who voted.
> >
> > -Taylor
> >
> > > On Mar 21, 2017, at 3:26 PM, P. Taylor Goetz 
> wrote:
> > >
> > > This is a call to vote on releasing Apache Storm 1.1.0 (rc3)
> > >
> > > Full list of changes in this release:
> > >
> > > https://git-wip-us.apache.org/repos/asf?p=storm.git;a=blob_
> > plain;f=CHANGELOG.md;h=68fbab3c4f91359bd397d93a157830542839b002;hb=
> > e40d213de7067f7d3aa4d4992b81890d8ed6ff31
> > >
> > > The tag/commit to be voted upon is v1.1.0:
> > >
> > > https://git-wip-us.apache.org/repos/asf?p=storm.git;a=tree;h=
> > 7fa62404feb6b86b3143c851b46237580720eb6b;hb=
> e40d213de7067f7d3aa4d4992b8189
> > 0d8ed6ff31
> > >
> > > The source archive being voted upon can be found here:
> > >
> > > https://dist.apache.org/repos/dist/dev/storm/apache-storm-1.
> > 1.0-rc3/apache-storm-1.1.0-src.tar.gz
> > >
> > > Other release files, signatures and digests can be found here:
> > >
> > > https://dist.apache.org/repos/dist/dev/storm/apache-storm-1.1.0-rc3/
> > >
> > > The release artifacts are signed with the following key:
> > >
> > > https://git-wip-us.apache.org/repos/asf?p=storm.git;a=blob_
> > plain;f=KEYS;hb=22b832708295fa2c15c4f3c70ac0d2bc6fded4bd
> > >
> > > The Nexus staging repository for this release is:
> > >
> > > https://repository.apache.org/content/repositories/orgapachestorm-1047
> > >
> > > Please vote on releasing this package as Apache Storm 1.1.0.
> > >
> > > When voting, please list the actions taken to verify the release.
> > >
> > > This vote will be open for at least 72 hours.
> > >
> > > [ ] +1 Release this package as Apache Storm 1.1.0
> > > [ ]  0 No opinion
> > > [ ] -1 Do not release this package because...
> > >
> > > Thanks to everyone who contributed to this release.
> > >
> > > -Taylor
> >
> >
>
>
> --
>
> *Open Source Solutions for Text Engineering*
>
> http://www.digitalpebble.com
> http://digitalpebble.blogspot.com/
> #digitalpebble 
>



-- 

*Open Source Solutions for Text Engineering*

http://www.digitalpebble.com
http://digitalpebble.blogspot.com/
#digitalpebble

[GitHub] storm issue #2033: [STORM-2431]the default blobstore.dir is storm.local.dir/...

2017-03-29 Thread HeartSaVioR

Github user HeartSaVioR commented on the issue:

https://github.com/apache/storm/pull/2033
  
No need to worry about build failure for document update. I'm stuck with 
other things, and also 1.1.0 vote is not announced so I would like to take a 
look at pending PRs after that.


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

Re: [RESULT] [VOTE] Release Apache Storm 1.1.0 (RC3)

2017-03-29 Thread Jungtaek Lim

Julien,

I can't see your post in RC3 thread, both my inbox and also mail archive.

http://mail-archives.apache.org/mod_mbox/storm-dev/201703.mbox/%3C0F8C6435-28BA-4D89-9818-C438575EAC1E%40apache.org%3E

Recently I experienced posting to storm-dev@ sometimes fails 'silently'.
After that I check mail archive to see it posted after sending mail.

Anyway, could you paste your vote mail again so that we can see why you
cast -1?

Thanks,
Jungtaek Lim (HeartSaVioR)

2017년 3월 29일 (수) 오후 5:20, Julien Nioche 님이
작성:

Hi,

For the record and not that it changes the outcome of the vote but I had
expressed a -1 on the 22th March.

Julien

On 28 March 2017 at 22:09, P. Taylor Goetz  wrote:

> This vote is now closed and passes with X binding +1 votes, X non-binding
> +1 votes, and no -1 or 0 votes.
>
> Vote tally (* indicates a binding vote):
>
> +1
> Jungtaek Lim*
> Arun Mahadevan*
> Bobby Evans*
> Harsha Chintalapani*
> Hugo Da Cruz Louro*
> Satish Duggana*
> Alexandre Vermeerbergen
> P. Taylor Goetz*
>
> I will release the artifacts, allow 24+ hours for mirrors to replicate,
> and announce the release.
>
> Thanks to all who voted.
>
> -Taylor
>
> > On Mar 21, 2017, at 3:26 PM, P. Taylor Goetz  wrote:
> >
> > This is a call to vote on releasing Apache Storm 1.1.0 (rc3)
> >
> > Full list of changes in this release:
> >
> > https://git-wip-us.apache.org/repos/asf?p=storm.git;a=blob_
> plain;f=CHANGELOG.md;h=68fbab3c4f91359bd397d93a157830542839b002;hb=
> e40d213de7067f7d3aa4d4992b81890d8ed6ff31
> >
> > The tag/commit to be voted upon is v1.1.0:
> >
> > https://git-wip-us.apache.org/repos/asf?p=storm.git;a=tree;h=
> 7fa62404feb6b86b3143c851b46237580720eb6b;hb=e40d213de7067f7d3aa4d4992b8189
> 0d8ed6ff31
> >
> > The source archive being voted upon can be found here:
> >
> > https://dist.apache.org/repos/dist/dev/storm/apache-storm-1.
> 1.0-rc3/apache-storm-1.1.0-src.tar.gz
> >
> > Other release files, signatures and digests can be found here:
> >
> > https://dist.apache.org/repos/dist/dev/storm/apache-storm-1.1.0-rc3/
> >
> > The release artifacts are signed with the following key:
> >
> > https://git-wip-us.apache.org/repos/asf?p=storm.git;a=blob_
> plain;f=KEYS;hb=22b832708295fa2c15c4f3c70ac0d2bc6fded4bd
> >
> > The Nexus staging repository for this release is:
> >
> > https://repository.apache.org/content/repositories/orgapachestorm-1047
> >
> > Please vote on releasing this package as Apache Storm 1.1.0.
> >
> > When voting, please list the actions taken to verify the release.
> >
> > This vote will be open for at least 72 hours.
> >
> > [ ] +1 Release this package as Apache Storm 1.1.0
> > [ ]  0 No opinion
> > [ ] -1 Do not release this package because...
> >
> > Thanks to everyone who contributed to this release.
> >
> > -Taylor
>
>


--

*Open Source Solutions for Text Engineering*

http://www.digitalpebble.com
http://digitalpebble.blogspot.com/
#digitalpebble

[GitHub] storm issue #2033: [STORM-2431]the default blobstore.dir is storm.local.dir/...

2017-03-29 Thread liu-zhaokun

Github user liu-zhaokun commented on the issue:

https://github.com/apache/storm/pull/2033
  
Hello, @HeartSaVioR  I am sorry to bother you.
The CI was failed,need I do anything to solve it?But I think there is 
nothing to cause it to fail.


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

Re: [RESULT] [VOTE] Release Apache Storm 1.1.0 (RC3)

2017-03-29 Thread Julien Nioche

Hi,

For the record and not that it changes the outcome of the vote but I had
expressed a -1 on the 22th March.

Julien

On 28 March 2017 at 22:09, P. Taylor Goetz  wrote:

> This vote is now closed and passes with X binding +1 votes, X non-binding
> +1 votes, and no -1 or 0 votes.
>
> Vote tally (* indicates a binding vote):
>
> +1
> Jungtaek Lim*
> Arun Mahadevan*
> Bobby Evans*
> Harsha Chintalapani*
> Hugo Da Cruz Louro*
> Satish Duggana*
> Alexandre Vermeerbergen
> P. Taylor Goetz*
>
> I will release the artifacts, allow 24+ hours for mirrors to replicate,
> and announce the release.
>
> Thanks to all who voted.
>
> -Taylor
>
> > On Mar 21, 2017, at 3:26 PM, P. Taylor Goetz  wrote:
> >
> > This is a call to vote on releasing Apache Storm 1.1.0 (rc3)
> >
> > Full list of changes in this release:
> >
> > https://git-wip-us.apache.org/repos/asf?p=storm.git;a=blob_
> plain;f=CHANGELOG.md;h=68fbab3c4f91359bd397d93a157830542839b002;hb=
> e40d213de7067f7d3aa4d4992b81890d8ed6ff31
> >
> > The tag/commit to be voted upon is v1.1.0:
> >
> > https://git-wip-us.apache.org/repos/asf?p=storm.git;a=tree;h=
> 7fa62404feb6b86b3143c851b46237580720eb6b;hb=e40d213de7067f7d3aa4d4992b8189
> 0d8ed6ff31
> >
> > The source archive being voted upon can be found here:
> >
> > https://dist.apache.org/repos/dist/dev/storm/apache-storm-1.
> 1.0-rc3/apache-storm-1.1.0-src.tar.gz
> >
> > Other release files, signatures and digests can be found here:
> >
> > https://dist.apache.org/repos/dist/dev/storm/apache-storm-1.1.0-rc3/
> >
> > The release artifacts are signed with the following key:
> >
> > https://git-wip-us.apache.org/repos/asf?p=storm.git;a=blob_
> plain;f=KEYS;hb=22b832708295fa2c15c4f3c70ac0d2bc6fded4bd
> >
> > The Nexus staging repository for this release is:
> >
> > https://repository.apache.org/content/repositories/orgapachestorm-1047
> >
> > Please vote on releasing this package as Apache Storm 1.1.0.
> >
> > When voting, please list the actions taken to verify the release.
> >
> > This vote will be open for at least 72 hours.
> >
> > [ ] +1 Release this package as Apache Storm 1.1.0
> > [ ]  0 No opinion
> > [ ] -1 Do not release this package because...
> >
> > Thanks to everyone who contributed to this release.
> >
> > -Taylor
>
>


-- 

*Open Source Solutions for Text Engineering*

http://www.digitalpebble.com
http://digitalpebble.blogspot.com/
#digitalpebble

[GitHub] storm issue #2026: Eventhub3

[GitHub] storm pull request #2026: Eventhub3

Re: [DISCUSS] Ideas for resolving storm-drpc-server compilation issue on IDE

Re: [RESULT] [VOTE] Release Apache Storm 1.1.0 (RC3)

[GitHub] storm issue #2033: [STORM-2431]the default blobstore.dir is storm.local.dir/...

Re: [RESULT] [VOTE] Release Apache Storm 1.1.0 (RC3)

[GitHub] storm issue #2033: [STORM-2431]the default blobstore.dir is storm.local.dir/...

Re: [RESULT] [VOTE] Release Apache Storm 1.1.0 (RC3)

8 matches

Site Navigation

Mail list logo

Footer information