[jira] [Commented] (ACCUMULO-4331) Make port configuration and allocation consistent across services
[
https://issues.apache.org/jira/browse/ACCUMULO-4331?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15319673#comment-15319673
]
Dave Marion commented on ACCUMULO-4331:
---
I was not thinking of doing anything with port search, other than making it use
a 1up algorithm instead of random. I was thinking of allowing a single port (N
or zero) or a range or ports (N-M) to be allowed in the port properties
mentioned in the description. For example:
{noformat}
tserver.port.client
9997
{noformat}
or
{noformat}
tserver.port.client
0
{noformat}
or
{noformat}
tserver.port.client
9997-
{noformat}
> Make port configuration and allocation consistent across services
> -
>
> Key: ACCUMULO-4331
> URL: https://issues.apache.org/jira/browse/ACCUMULO-4331
> Project: Accumulo
> Issue Type: Bug
>Affects Versions: 1.8.0
>Reporter: Dave Marion
> Fix For: 1.8.0
>
>
> There was some discussion in ACCUMULO-4328 about ports, so I decided to track
> down how the client ports are configured and allocated. Issues raised in the
> discussion were:
> 1. The port search feature was not well understood
> 2. Ephemeral port allocation makes it hard to lock servers down (e.g.
> iptables)
> Looking through the code I found the following properties allocate a port
> number based on conf.getPort(). This returns the port number based on the
> property and supports either a single value or zero. Then, in the server
> component (monitor, tracer, gc, etc) this value is used when creating a
> ServerSocket. If the port is already in use, the process will fail.
> {noformat}
> monitor.port.log4j
> trace.port.client
> gc.port.client
> monitor.port.client
> {noformat}
> The following properties use TServerUtils.startServer which uses the value in
> the property to start the TServer. If the value is zero, then it picks a
> random port between 1024 and 65535. If tserver.port.search is enabled, then
> it will try a thousand times to bind to a random port.
> {noformat}
> tserver.port.client
> master.port.client
> master.replication.coordinator.port
> replication.receipt.service.port
> {noformat}
> I'm proposing that we deprecate the tserver.port.search property and the
> value zero in the property value for the properties above. Instead, I think
> we should allow the user to specify a single value or a range (M-N).
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (ACCUMULO-4331) Make port configuration and allocation consistent across services
[
https://issues.apache.org/jira/browse/ACCUMULO-4331?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15319651#comment-15319651
]
Keith Turner commented on ACCUMULO-4331:
bq. Allow the user to specify a range of ports to be used (M-N). [new feature]
How are you thinking of doing this? With a tserver.port.search.max prop?
> Make port configuration and allocation consistent across services
> -
>
> Key: ACCUMULO-4331
> URL: https://issues.apache.org/jira/browse/ACCUMULO-4331
> Project: Accumulo
> Issue Type: Bug
>Affects Versions: 1.8.0
>Reporter: Dave Marion
> Fix For: 1.8.0
>
>
> There was some discussion in ACCUMULO-4328 about ports, so I decided to track
> down how the client ports are configured and allocated. Issues raised in the
> discussion were:
> 1. The port search feature was not well understood
> 2. Ephemeral port allocation makes it hard to lock servers down (e.g.
> iptables)
> Looking through the code I found the following properties allocate a port
> number based on conf.getPort(). This returns the port number based on the
> property and supports either a single value or zero. Then, in the server
> component (monitor, tracer, gc, etc) this value is used when creating a
> ServerSocket. If the port is already in use, the process will fail.
> {noformat}
> monitor.port.log4j
> trace.port.client
> gc.port.client
> monitor.port.client
> {noformat}
> The following properties use TServerUtils.startServer which uses the value in
> the property to start the TServer. If the value is zero, then it picks a
> random port between 1024 and 65535. If tserver.port.search is enabled, then
> it will try a thousand times to bind to a random port.
> {noformat}
> tserver.port.client
> master.port.client
> master.replication.coordinator.port
> replication.receipt.service.port
> {noformat}
> I'm proposing that we deprecate the tserver.port.search property and the
> value zero in the property value for the properties above. Instead, I think
> we should allow the user to specify a single value or a range (M-N).
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Comment Edited] (ACCUMULO-4331) Make port configuration and allocation consistent across services
[
https://issues.apache.org/jira/browse/ACCUMULO-4331?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15319456#comment-15319456
]
Dave Marion edited comment on ACCUMULO-4331 at 6/7/16 9:27 PM:
---
So, based on the comments here, it seems the consensus is to
1. Keep zero as a valid port option, which means any ephemeral port
2. Keep tserver.port.search, but maybe change it such that it performs a 1-up
attempt instead of random
3. Allow the user to specify a range of ports to be used (M-N). [new feature]
4. If user specifies a range of ports, and tserver.port.search is disabled, any
port in that range is still valid.
5. If user specifies a range of ports, and tserver.port.search is enabled, we
must try all the ports in the range before searching
was (Author: dlmarion):
So, based on the comments here, it seems the consensus is to
1. Keep zero as a valid port option, which means any ephemeral port
2. Keep tserver.port.search, but maybe change it such that it performs a 1-up
attempt instead of random
3. Allow the user to specify a range of ports to be used (M-N). [new feature]
4. If user specifies a range of ports, and tserver.port.search is disabled, any
port in that range is still valid.
> Make port configuration and allocation consistent across services
> -
>
> Key: ACCUMULO-4331
> URL: https://issues.apache.org/jira/browse/ACCUMULO-4331
> Project: Accumulo
> Issue Type: Bug
>Affects Versions: 1.8.0
>Reporter: Dave Marion
> Fix For: 1.8.0
>
>
> There was some discussion in ACCUMULO-4328 about ports, so I decided to track
> down how the client ports are configured and allocated. Issues raised in the
> discussion were:
> 1. The port search feature was not well understood
> 2. Ephemeral port allocation makes it hard to lock servers down (e.g.
> iptables)
> Looking through the code I found the following properties allocate a port
> number based on conf.getPort(). This returns the port number based on the
> property and supports either a single value or zero. Then, in the server
> component (monitor, tracer, gc, etc) this value is used when creating a
> ServerSocket. If the port is already in use, the process will fail.
> {noformat}
> monitor.port.log4j
> trace.port.client
> gc.port.client
> monitor.port.client
> {noformat}
> The following properties use TServerUtils.startServer which uses the value in
> the property to start the TServer. If the value is zero, then it picks a
> random port between 1024 and 65535. If tserver.port.search is enabled, then
> it will try a thousand times to bind to a random port.
> {noformat}
> tserver.port.client
> master.port.client
> master.replication.coordinator.port
> replication.receipt.service.port
> {noformat}
> I'm proposing that we deprecate the tserver.port.search property and the
> value zero in the property value for the properties above. Instead, I think
> we should allow the user to specify a single value or a range (M-N).
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Comment Edited] (ACCUMULO-4331) Make port configuration and allocation consistent across services
[
https://issues.apache.org/jira/browse/ACCUMULO-4331?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15319456#comment-15319456
]
Dave Marion edited comment on ACCUMULO-4331 at 6/7/16 9:24 PM:
---
So, based on the comments here, it seems the consensus is to
1. Keep zero as a valid port option, which means any ephemeral port
2. Keep tserver.port.search, but maybe change it such that it performs a 1-up
attempt instead of random
3. Allow the user to specify a range of ports to be used (M-N). [new feature]
4. If user specifies a range of ports, and tserver.port.search is disabled, any
port in that range is still valid.
was (Author: dlmarion):
So, based on the comments here, it seems the consensus is to
1. Keep zero as a valid port option, which means any ephemeral port
2. Keep tserver.port.search, but maybe change it such that it performs a 1-up
attempt instead of random
3. Allow the user to specify a range of ports to be used (M-N).
I'll add a 4th here assuming that I am going to merge in ACCUMULO-4328:
4. IF the system property "instance" is set to N, then the Nth port in the
tserver.client.port range will be used. This is an attempt to provide some port
stability to multiple tservers on the same host.
> Make port configuration and allocation consistent across services
> -
>
> Key: ACCUMULO-4331
> URL: https://issues.apache.org/jira/browse/ACCUMULO-4331
> Project: Accumulo
> Issue Type: Bug
>Affects Versions: 1.8.0
>Reporter: Dave Marion
> Fix For: 1.8.0
>
>
> There was some discussion in ACCUMULO-4328 about ports, so I decided to track
> down how the client ports are configured and allocated. Issues raised in the
> discussion were:
> 1. The port search feature was not well understood
> 2. Ephemeral port allocation makes it hard to lock servers down (e.g.
> iptables)
> Looking through the code I found the following properties allocate a port
> number based on conf.getPort(). This returns the port number based on the
> property and supports either a single value or zero. Then, in the server
> component (monitor, tracer, gc, etc) this value is used when creating a
> ServerSocket. If the port is already in use, the process will fail.
> {noformat}
> monitor.port.log4j
> trace.port.client
> gc.port.client
> monitor.port.client
> {noformat}
> The following properties use TServerUtils.startServer which uses the value in
> the property to start the TServer. If the value is zero, then it picks a
> random port between 1024 and 65535. If tserver.port.search is enabled, then
> it will try a thousand times to bind to a random port.
> {noformat}
> tserver.port.client
> master.port.client
> master.replication.coordinator.port
> replication.receipt.service.port
> {noformat}
> I'm proposing that we deprecate the tserver.port.search property and the
> value zero in the property value for the properties above. Instead, I think
> we should allow the user to specify a single value or a range (M-N).
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (ACCUMULO-4331) Make port configuration and allocation consistent across services
[
https://issues.apache.org/jira/browse/ACCUMULO-4331?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15319456#comment-15319456
]
Dave Marion commented on ACCUMULO-4331:
---
So, based on the comments here, it seems the consensus is to
1. Keep zero as a valid port option, which means any ephemeral port
2. Keep tserver.port.search, but maybe change it such that it performs a 1-up
attempt instead of random
3. Allow the user to specify a range of ports to be used (M-N).
I'll add a 4th here assuming that I am going to merge in ACCUMULO-4328:
4. IF the system property "instance" is set to N, then the Nth port in the
tserver.client.port range will be used. This is an attempt to provide some port
stability to multiple tservers on the same host.
> Make port configuration and allocation consistent across services
> -
>
> Key: ACCUMULO-4331
> URL: https://issues.apache.org/jira/browse/ACCUMULO-4331
> Project: Accumulo
> Issue Type: Bug
>Affects Versions: 1.8.0
>Reporter: Dave Marion
> Fix For: 1.8.0
>
>
> There was some discussion in ACCUMULO-4328 about ports, so I decided to track
> down how the client ports are configured and allocated. Issues raised in the
> discussion were:
> 1. The port search feature was not well understood
> 2. Ephemeral port allocation makes it hard to lock servers down (e.g.
> iptables)
> Looking through the code I found the following properties allocate a port
> number based on conf.getPort(). This returns the port number based on the
> property and supports either a single value or zero. Then, in the server
> component (monitor, tracer, gc, etc) this value is used when creating a
> ServerSocket. If the port is already in use, the process will fail.
> {noformat}
> monitor.port.log4j
> trace.port.client
> gc.port.client
> monitor.port.client
> {noformat}
> The following properties use TServerUtils.startServer which uses the value in
> the property to start the TServer. If the value is zero, then it picks a
> random port between 1024 and 65535. If tserver.port.search is enabled, then
> it will try a thousand times to bind to a random port.
> {noformat}
> tserver.port.client
> master.port.client
> master.replication.coordinator.port
> replication.receipt.service.port
> {noformat}
> I'm proposing that we deprecate the tserver.port.search property and the
> value zero in the property value for the properties above. Instead, I think
> we should allow the user to specify a single value or a range (M-N).
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (ACCUMULO-4331) Make port configuration and allocation consistent across services
[
https://issues.apache.org/jira/browse/ACCUMULO-4331?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15319373#comment-15319373
]
Josh Elser commented on ACCUMULO-4331:
--
bq. So, you don't think we should deprecate tserver.port.search in favor of a
port range? We should fix it instead?
I think making portsearch bound to a range would be much more useful. I think
it's a rather obtuse configuration property now -- I'd be surprised if anyone
actually dug into how it works and is relying on it (even more so after Mike
found that it doesn't even work right).
> Make port configuration and allocation consistent across services
> -
>
> Key: ACCUMULO-4331
> URL: https://issues.apache.org/jira/browse/ACCUMULO-4331
> Project: Accumulo
> Issue Type: Bug
>Affects Versions: 1.8.0
>Reporter: Dave Marion
> Fix For: 1.8.0
>
>
> There was some discussion in ACCUMULO-4328 about ports, so I decided to track
> down how the client ports are configured and allocated. Issues raised in the
> discussion were:
> 1. The port search feature was not well understood
> 2. Ephemeral port allocation makes it hard to lock servers down (e.g.
> iptables)
> Looking through the code I found the following properties allocate a port
> number based on conf.getPort(). This returns the port number based on the
> property and supports either a single value or zero. Then, in the server
> component (monitor, tracer, gc, etc) this value is used when creating a
> ServerSocket. If the port is already in use, the process will fail.
> {noformat}
> monitor.port.log4j
> trace.port.client
> gc.port.client
> monitor.port.client
> {noformat}
> The following properties use TServerUtils.startServer which uses the value in
> the property to start the TServer. If the value is zero, then it picks a
> random port between 1024 and 65535. If tserver.port.search is enabled, then
> it will try a thousand times to bind to a random port.
> {noformat}
> tserver.port.client
> master.port.client
> master.replication.coordinator.port
> replication.receipt.service.port
> {noformat}
> I'm proposing that we deprecate the tserver.port.search property and the
> value zero in the property value for the properties above. Instead, I think
> we should allow the user to specify a single value or a range (M-N).
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (ACCUMULO-4331) Make port configuration and allocation consistent across services
[
https://issues.apache.org/jira/browse/ACCUMULO-4331?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15319367#comment-15319367
]
Dave Marion commented on ACCUMULO-4331:
---
In my discussions with [~ctubbsii] regarding this, he feels the same way.
However, I think that is a different JIRA as I think that is much more work
than what I am proposing here. You would have to move the port handling code
out of Java and into the shell scripts.
> Make port configuration and allocation consistent across services
> -
>
> Key: ACCUMULO-4331
> URL: https://issues.apache.org/jira/browse/ACCUMULO-4331
> Project: Accumulo
> Issue Type: Bug
>Affects Versions: 1.8.0
>Reporter: Dave Marion
> Fix For: 1.8.0
>
>
> There was some discussion in ACCUMULO-4328 about ports, so I decided to track
> down how the client ports are configured and allocated. Issues raised in the
> discussion were:
> 1. The port search feature was not well understood
> 2. Ephemeral port allocation makes it hard to lock servers down (e.g.
> iptables)
> Looking through the code I found the following properties allocate a port
> number based on conf.getPort(). This returns the port number based on the
> property and supports either a single value or zero. Then, in the server
> component (monitor, tracer, gc, etc) this value is used when creating a
> ServerSocket. If the port is already in use, the process will fail.
> {noformat}
> monitor.port.log4j
> trace.port.client
> gc.port.client
> monitor.port.client
> {noformat}
> The following properties use TServerUtils.startServer which uses the value in
> the property to start the TServer. If the value is zero, then it picks a
> random port between 1024 and 65535. If tserver.port.search is enabled, then
> it will try a thousand times to bind to a random port.
> {noformat}
> tserver.port.client
> master.port.client
> master.replication.coordinator.port
> replication.receipt.service.port
> {noformat}
> I'm proposing that we deprecate the tserver.port.search property and the
> value zero in the property value for the properties above. Instead, I think
> we should allow the user to specify a single value or a range (M-N).
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (ACCUMULO-4331) Make port configuration and allocation consistent across services
[
https://issues.apache.org/jira/browse/ACCUMULO-4331?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15319365#comment-15319365
]
Dave Marion commented on ACCUMULO-4331:
---
So, you don't think we should deprecate tserver.port.search in favor of a port
range? We should fix it instead?
> Make port configuration and allocation consistent across services
> -
>
> Key: ACCUMULO-4331
> URL: https://issues.apache.org/jira/browse/ACCUMULO-4331
> Project: Accumulo
> Issue Type: Bug
>Affects Versions: 1.8.0
>Reporter: Dave Marion
> Fix For: 1.8.0
>
>
> There was some discussion in ACCUMULO-4328 about ports, so I decided to track
> down how the client ports are configured and allocated. Issues raised in the
> discussion were:
> 1. The port search feature was not well understood
> 2. Ephemeral port allocation makes it hard to lock servers down (e.g.
> iptables)
> Looking through the code I found the following properties allocate a port
> number based on conf.getPort(). This returns the port number based on the
> property and supports either a single value or zero. Then, in the server
> component (monitor, tracer, gc, etc) this value is used when creating a
> ServerSocket. If the port is already in use, the process will fail.
> {noformat}
> monitor.port.log4j
> trace.port.client
> gc.port.client
> monitor.port.client
> {noformat}
> The following properties use TServerUtils.startServer which uses the value in
> the property to start the TServer. If the value is zero, then it picks a
> random port between 1024 and 65535. If tserver.port.search is enabled, then
> it will try a thousand times to bind to a random port.
> {noformat}
> tserver.port.client
> master.port.client
> master.replication.coordinator.port
> replication.receipt.service.port
> {noformat}
> I'm proposing that we deprecate the tserver.port.search property and the
> value zero in the property value for the properties above. Instead, I think
> we should allow the user to specify a single value or a range (M-N).
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (ACCUMULO-4331) Make port configuration and allocation consistent across services
[
https://issues.apache.org/jira/browse/ACCUMULO-4331?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15319282#comment-15319282
]
Mike Walch commented on ACCUMULO-4331:
--
I don't think tserver.port.search works correctly. If it's not deprecated it
should be fixed. I set it to true and and set tserver.client.port to 9997.
The first port that I was given was 9997 but the next port was 32843. I think
a random port is being chosen rather than a 1 up. I am using 1.7.1 btw.
> Make port configuration and allocation consistent across services
> -
>
> Key: ACCUMULO-4331
> URL: https://issues.apache.org/jira/browse/ACCUMULO-4331
> Project: Accumulo
> Issue Type: Bug
>Affects Versions: 1.8.0
>Reporter: Dave Marion
> Fix For: 1.8.0
>
>
> There was some discussion in ACCUMULO-4328 about ports, so I decided to track
> down how the client ports are configured and allocated. Issues raised in the
> discussion were:
> 1. The port search feature was not well understood
> 2. Ephemeral port allocation makes it hard to lock servers down (e.g.
> iptables)
> Looking through the code I found the following properties allocate a port
> number based on conf.getPort(). This returns the port number based on the
> property and supports either a single value or zero. Then, in the server
> component (monitor, tracer, gc, etc) this value is used when creating a
> ServerSocket. If the port is already in use, the process will fail.
> {noformat}
> monitor.port.log4j
> trace.port.client
> gc.port.client
> monitor.port.client
> {noformat}
> The following properties use TServerUtils.startServer which uses the value in
> the property to start the TServer. If the value is zero, then it picks a
> random port between 1024 and 65535. If tserver.port.search is enabled, then
> it will try a thousand times to bind to a random port.
> {noformat}
> tserver.port.client
> master.port.client
> master.replication.coordinator.port
> replication.receipt.service.port
> {noformat}
> I'm proposing that we deprecate the tserver.port.search property and the
> value zero in the property value for the properties above. Instead, I think
> we should allow the user to specify a single value or a range (M-N).
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (ACCUMULO-4331) Make port configuration and allocation consistent across services
[
https://issues.apache.org/jira/browse/ACCUMULO-4331?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15319276#comment-15319276
]
Mike Walch commented on ACCUMULO-4331:
--
Agree. random is not the same as 1 up
> Make port configuration and allocation consistent across services
> -
>
> Key: ACCUMULO-4331
> URL: https://issues.apache.org/jira/browse/ACCUMULO-4331
> Project: Accumulo
> Issue Type: Bug
>Affects Versions: 1.8.0
>Reporter: Dave Marion
> Fix For: 1.8.0
>
>
> There was some discussion in ACCUMULO-4328 about ports, so I decided to track
> down how the client ports are configured and allocated. Issues raised in the
> discussion were:
> 1. The port search feature was not well understood
> 2. Ephemeral port allocation makes it hard to lock servers down (e.g.
> iptables)
> Looking through the code I found the following properties allocate a port
> number based on conf.getPort(). This returns the port number based on the
> property and supports either a single value or zero. Then, in the server
> component (monitor, tracer, gc, etc) this value is used when creating a
> ServerSocket. If the port is already in use, the process will fail.
> {noformat}
> monitor.port.log4j
> trace.port.client
> gc.port.client
> monitor.port.client
> {noformat}
> The following properties use TServerUtils.startServer which uses the value in
> the property to start the TServer. If the value is zero, then it picks a
> random port between 1024 and 65535. If tserver.port.search is enabled, then
> it will try a thousand times to bind to a random port.
> {noformat}
> tserver.port.client
> master.port.client
> master.replication.coordinator.port
> replication.receipt.service.port
> {noformat}
> I'm proposing that we deprecate the tserver.port.search property and the
> value zero in the property value for the properties above. Instead, I think
> we should allow the user to specify a single value or a range (M-N).
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Comment Edited] (ACCUMULO-4331) Make port configuration and allocation consistent across services
[
https://issues.apache.org/jira/browse/ACCUMULO-4331?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15319247#comment-15319247
]
Michael Wall edited comment on ACCUMULO-4331 at 6/7/16 8:02 PM:
Yeah, that is what I am suggesting I guess. Adding the ability to pass in the
ports when start-server.sh is called. If 1 or more are there, iterate over the
port, set -Dport, start the server on that port and use the port in the log
name.
was (Author: mjwall):
Yeah, that is what I am suggesting I guess.
> Make port configuration and allocation consistent across services
> -
>
> Key: ACCUMULO-4331
> URL: https://issues.apache.org/jira/browse/ACCUMULO-4331
> Project: Accumulo
> Issue Type: Bug
>Affects Versions: 1.8.0
>Reporter: Dave Marion
> Fix For: 1.8.0
>
>
> There was some discussion in ACCUMULO-4328 about ports, so I decided to track
> down how the client ports are configured and allocated. Issues raised in the
> discussion were:
> 1. The port search feature was not well understood
> 2. Ephemeral port allocation makes it hard to lock servers down (e.g.
> iptables)
> Looking through the code I found the following properties allocate a port
> number based on conf.getPort(). This returns the port number based on the
> property and supports either a single value or zero. Then, in the server
> component (monitor, tracer, gc, etc) this value is used when creating a
> ServerSocket. If the port is already in use, the process will fail.
> {noformat}
> monitor.port.log4j
> trace.port.client
> gc.port.client
> monitor.port.client
> {noformat}
> The following properties use TServerUtils.startServer which uses the value in
> the property to start the TServer. If the value is zero, then it picks a
> random port between 1024 and 65535. If tserver.port.search is enabled, then
> it will try a thousand times to bind to a random port.
> {noformat}
> tserver.port.client
> master.port.client
> master.replication.coordinator.port
> replication.receipt.service.port
> {noformat}
> I'm proposing that we deprecate the tserver.port.search property and the
> value zero in the property value for the properties above. Instead, I think
> we should allow the user to specify a single value or a range (M-N).
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (ACCUMULO-4331) Make port configuration and allocation consistent across services
[
https://issues.apache.org/jira/browse/ACCUMULO-4331?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15319247#comment-15319247
]
Michael Wall commented on ACCUMULO-4331:
Yeah, that is what I am suggesting I guess.
> Make port configuration and allocation consistent across services
> -
>
> Key: ACCUMULO-4331
> URL: https://issues.apache.org/jira/browse/ACCUMULO-4331
> Project: Accumulo
> Issue Type: Bug
>Affects Versions: 1.8.0
>Reporter: Dave Marion
> Fix For: 1.8.0
>
>
> There was some discussion in ACCUMULO-4328 about ports, so I decided to track
> down how the client ports are configured and allocated. Issues raised in the
> discussion were:
> 1. The port search feature was not well understood
> 2. Ephemeral port allocation makes it hard to lock servers down (e.g.
> iptables)
> Looking through the code I found the following properties allocate a port
> number based on conf.getPort(). This returns the port number based on the
> property and supports either a single value or zero. Then, in the server
> component (monitor, tracer, gc, etc) this value is used when creating a
> ServerSocket. If the port is already in use, the process will fail.
> {noformat}
> monitor.port.log4j
> trace.port.client
> gc.port.client
> monitor.port.client
> {noformat}
> The following properties use TServerUtils.startServer which uses the value in
> the property to start the TServer. If the value is zero, then it picks a
> random port between 1024 and 65535. If tserver.port.search is enabled, then
> it will try a thousand times to bind to a random port.
> {noformat}
> tserver.port.client
> master.port.client
> master.replication.coordinator.port
> replication.receipt.service.port
> {noformat}
> I'm proposing that we deprecate the tserver.port.search property and the
> value zero in the property value for the properties above. Instead, I think
> we should allow the user to specify a single value or a range (M-N).
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
Accumulo-Pull-Requests - Build # 323 - Fixed
The Apache Jenkins build system has built Accumulo-Pull-Requests (build #323) Status: Fixed Check console output at https://builds.apache.org/job/Accumulo-Pull-Requests/323/ to view the results.
[jira] [Commented] (ACCUMULO-4331) Make port configuration and allocation consistent across services
[
https://issues.apache.org/jira/browse/ACCUMULO-4331?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15319043#comment-15319043
]
Dave Marion commented on ACCUMULO-4331:
---
If we set tserver.port.client to "9997,9998," or "9997-" then any one
of the three tservers in your example would allocate any of the three ports in
the range specified. Do we want to make the tablet server code aware that there
could be multiple instances on a host, that it is the Nth instance, and it must
allocate the Nth port in the range?
> Make port configuration and allocation consistent across services
> -
>
> Key: ACCUMULO-4331
> URL: https://issues.apache.org/jira/browse/ACCUMULO-4331
> Project: Accumulo
> Issue Type: Bug
>Affects Versions: 1.8.0
>Reporter: Dave Marion
> Fix For: 1.8.0
>
>
> There was some discussion in ACCUMULO-4328 about ports, so I decided to track
> down how the client ports are configured and allocated. Issues raised in the
> discussion were:
> 1. The port search feature was not well understood
> 2. Ephemeral port allocation makes it hard to lock servers down (e.g.
> iptables)
> Looking through the code I found the following properties allocate a port
> number based on conf.getPort(). This returns the port number based on the
> property and supports either a single value or zero. Then, in the server
> component (monitor, tracer, gc, etc) this value is used when creating a
> ServerSocket. If the port is already in use, the process will fail.
> {noformat}
> monitor.port.log4j
> trace.port.client
> gc.port.client
> monitor.port.client
> {noformat}
> The following properties use TServerUtils.startServer which uses the value in
> the property to start the TServer. If the value is zero, then it picks a
> random port between 1024 and 65535. If tserver.port.search is enabled, then
> it will try a thousand times to bind to a random port.
> {noformat}
> tserver.port.client
> master.port.client
> master.replication.coordinator.port
> replication.receipt.service.port
> {noformat}
> I'm proposing that we deprecate the tserver.port.search property and the
> value zero in the property value for the properties above. Instead, I think
> we should allow the user to specify a single value or a range (M-N).
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (ACCUMULO-4331) Make port configuration and allocation consistent across services
[
https://issues.apache.org/jira/browse/ACCUMULO-4331?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15319037#comment-15319037
]
Dave Marion commented on ACCUMULO-4331:
---
I don't know that we can do what you are suggesting in the manner in which you
are suggesting it. You would have to move the port allocation code out of Java
and into the shell scripts. Another approach would be to push the port into the
Log4J MDC and then add the MDC key to the logging pattern. I had taken this
approach, but using the PID, in the PR against 1.6.x
> Make port configuration and allocation consistent across services
> -
>
> Key: ACCUMULO-4331
> URL: https://issues.apache.org/jira/browse/ACCUMULO-4331
> Project: Accumulo
> Issue Type: Bug
>Affects Versions: 1.8.0
>Reporter: Dave Marion
> Fix For: 1.8.0
>
>
> There was some discussion in ACCUMULO-4328 about ports, so I decided to track
> down how the client ports are configured and allocated. Issues raised in the
> discussion were:
> 1. The port search feature was not well understood
> 2. Ephemeral port allocation makes it hard to lock servers down (e.g.
> iptables)
> Looking through the code I found the following properties allocate a port
> number based on conf.getPort(). This returns the port number based on the
> property and supports either a single value or zero. Then, in the server
> component (monitor, tracer, gc, etc) this value is used when creating a
> ServerSocket. If the port is already in use, the process will fail.
> {noformat}
> monitor.port.log4j
> trace.port.client
> gc.port.client
> monitor.port.client
> {noformat}
> The following properties use TServerUtils.startServer which uses the value in
> the property to start the TServer. If the value is zero, then it picks a
> random port between 1024 and 65535. If tserver.port.search is enabled, then
> it will try a thousand times to bind to a random port.
> {noformat}
> tserver.port.client
> master.port.client
> master.replication.coordinator.port
> replication.receipt.service.port
> {noformat}
> I'm proposing that we deprecate the tserver.port.search property and the
> value zero in the property value for the properties above. Instead, I think
> we should allow the user to specify a single value or a range (M-N).
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (ACCUMULO-4331) Make port configuration and allocation consistent across services
[
https://issues.apache.org/jira/browse/ACCUMULO-4331?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15319030#comment-15319030
]
Dave Marion commented on ACCUMULO-4331:
---
And note that 0 is totally random, not a 1 up attempt until it succeeds
> Make port configuration and allocation consistent across services
> -
>
> Key: ACCUMULO-4331
> URL: https://issues.apache.org/jira/browse/ACCUMULO-4331
> Project: Accumulo
> Issue Type: Bug
>Affects Versions: 1.8.0
>Reporter: Dave Marion
> Fix For: 1.8.0
>
>
> There was some discussion in ACCUMULO-4328 about ports, so I decided to track
> down how the client ports are configured and allocated. Issues raised in the
> discussion were:
> 1. The port search feature was not well understood
> 2. Ephemeral port allocation makes it hard to lock servers down (e.g.
> iptables)
> Looking through the code I found the following properties allocate a port
> number based on conf.getPort(). This returns the port number based on the
> property and supports either a single value or zero. Then, in the server
> component (monitor, tracer, gc, etc) this value is used when creating a
> ServerSocket. If the port is already in use, the process will fail.
> {noformat}
> monitor.port.log4j
> trace.port.client
> gc.port.client
> monitor.port.client
> {noformat}
> The following properties use TServerUtils.startServer which uses the value in
> the property to start the TServer. If the value is zero, then it picks a
> random port between 1024 and 65535. If tserver.port.search is enabled, then
> it will try a thousand times to bind to a random port.
> {noformat}
> tserver.port.client
> master.port.client
> master.replication.coordinator.port
> replication.receipt.service.port
> {noformat}
> I'm proposing that we deprecate the tserver.port.search property and the
> value zero in the property value for the properties above. Instead, I think
> we should allow the user to specify a single value or a range (M-N).
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (ACCUMULO-4331) Make port configuration and allocation consistent across services
[
https://issues.apache.org/jira/browse/ACCUMULO-4331?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15319029#comment-15319029
]
Dave Marion commented on ACCUMULO-4331:
---
The instance is in the log file name in the PR.
> Make port configuration and allocation consistent across services
> -
>
> Key: ACCUMULO-4331
> URL: https://issues.apache.org/jira/browse/ACCUMULO-4331
> Project: Accumulo
> Issue Type: Bug
>Affects Versions: 1.8.0
>Reporter: Dave Marion
> Fix For: 1.8.0
>
>
> There was some discussion in ACCUMULO-4328 about ports, so I decided to track
> down how the client ports are configured and allocated. Issues raised in the
> discussion were:
> 1. The port search feature was not well understood
> 2. Ephemeral port allocation makes it hard to lock servers down (e.g.
> iptables)
> Looking through the code I found the following properties allocate a port
> number based on conf.getPort(). This returns the port number based on the
> property and supports either a single value or zero. Then, in the server
> component (monitor, tracer, gc, etc) this value is used when creating a
> ServerSocket. If the port is already in use, the process will fail.
> {noformat}
> monitor.port.log4j
> trace.port.client
> gc.port.client
> monitor.port.client
> {noformat}
> The following properties use TServerUtils.startServer which uses the value in
> the property to start the TServer. If the value is zero, then it picks a
> random port between 1024 and 65535. If tserver.port.search is enabled, then
> it will try a thousand times to bind to a random port.
> {noformat}
> tserver.port.client
> master.port.client
> master.replication.coordinator.port
> replication.receipt.service.port
> {noformat}
> I'm proposing that we deprecate the tserver.port.search property and the
> value zero in the property value for the properties above. Instead, I think
> we should allow the user to specify a single value or a range (M-N).
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (ACCUMULO-4331) Make port configuration and allocation consistent across services
[
https://issues.apache.org/jira/browse/ACCUMULO-4331?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15318982#comment-15318982
]
Keith Turner commented on ACCUMULO-4331:
That's an interesting observation. Who would have thought that 0=1024-65553
> Make port configuration and allocation consistent across services
> -
>
> Key: ACCUMULO-4331
> URL: https://issues.apache.org/jira/browse/ACCUMULO-4331
> Project: Accumulo
> Issue Type: Bug
>Affects Versions: 1.8.0
>Reporter: Dave Marion
> Fix For: 1.8.0
>
>
> There was some discussion in ACCUMULO-4328 about ports, so I decided to track
> down how the client ports are configured and allocated. Issues raised in the
> discussion were:
> 1. The port search feature was not well understood
> 2. Ephemeral port allocation makes it hard to lock servers down (e.g.
> iptables)
> Looking through the code I found the following properties allocate a port
> number based on conf.getPort(). This returns the port number based on the
> property and supports either a single value or zero. Then, in the server
> component (monitor, tracer, gc, etc) this value is used when creating a
> ServerSocket. If the port is already in use, the process will fail.
> {noformat}
> monitor.port.log4j
> trace.port.client
> gc.port.client
> monitor.port.client
> {noformat}
> The following properties use TServerUtils.startServer which uses the value in
> the property to start the TServer. If the value is zero, then it picks a
> random port between 1024 and 65535. If tserver.port.search is enabled, then
> it will try a thousand times to bind to a random port.
> {noformat}
> tserver.port.client
> master.port.client
> master.replication.coordinator.port
> replication.receipt.service.port
> {noformat}
> I'm proposing that we deprecate the tserver.port.search property and the
> value zero in the property value for the properties above. Instead, I think
> we should allow the user to specify a single value or a range (M-N).
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (ACCUMULO-4330) ZooKeeper test logs way too big
[ https://issues.apache.org/jira/browse/ACCUMULO-4330?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15318970#comment-15318970 ] Keith Turner commented on ACCUMULO-4330: Nice find. bq. MiniAccumuloCluster probably should not write the DEBUG logs for ZooKeeper by default. No, it should not default to that for ZK. ZK is too verbose at INFO. > ZooKeeper test logs way too big > --- > > Key: ACCUMULO-4330 > URL: https://issues.apache.org/jira/browse/ACCUMULO-4330 > Project: Accumulo > Issue Type: Bug > Components: test >Reporter: Christopher Tubbs >Priority: Critical > Fix For: 1.6.6, 1.7.2, 1.8.0 > > > Could not complete a run of the ITs on Jenkins because the > ZooKeeperServerMain_\*.out files were far too big. > The ZooKeeperServerMain log file for > org.apache.accumulo.harness.SharedMiniClusterBase grew to over 17G before > filling up my disk. > MiniAccumuloCluster probably should not write the DEBUG logs for ZooKeeper by > default. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (ACCUMULO-4331) Make port configuration and allocation consistent across services
[
https://issues.apache.org/jira/browse/ACCUMULO-4331?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15318954#comment-15318954
]
Mike Walch commented on ACCUMULO-4331:
--
It would be great if the 0 option was kept but users could specify a single
value or a range (M-N). The 0 option is just shorthand for the range
1024-65535.
> Make port configuration and allocation consistent across services
> -
>
> Key: ACCUMULO-4331
> URL: https://issues.apache.org/jira/browse/ACCUMULO-4331
> Project: Accumulo
> Issue Type: Bug
>Affects Versions: 1.8.0
>Reporter: Dave Marion
> Fix For: 1.8.0
>
>
> There was some discussion in ACCUMULO-4328 about ports, so I decided to track
> down how the client ports are configured and allocated. Issues raised in the
> discussion were:
> 1. The port search feature was not well understood
> 2. Ephemeral port allocation makes it hard to lock servers down (e.g.
> iptables)
> Looking through the code I found the following properties allocate a port
> number based on conf.getPort(). This returns the port number based on the
> property and supports either a single value or zero. Then, in the server
> component (monitor, tracer, gc, etc) this value is used when creating a
> ServerSocket. If the port is already in use, the process will fail.
> {noformat}
> monitor.port.log4j
> trace.port.client
> gc.port.client
> monitor.port.client
> {noformat}
> The following properties use TServerUtils.startServer which uses the value in
> the property to start the TServer. If the value is zero, then it picks a
> random port between 1024 and 65535. If tserver.port.search is enabled, then
> it will try a thousand times to bind to a random port.
> {noformat}
> tserver.port.client
> master.port.client
> master.replication.coordinator.port
> replication.receipt.service.port
> {noformat}
> I'm proposing that we deprecate the tserver.port.search property and the
> value zero in the property value for the properties above. Instead, I think
> we should allow the user to specify a single value or a range (M-N).
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (ACCUMULO-4331) Make port configuration and allocation consistent across services
[
https://issues.apache.org/jira/browse/ACCUMULO-4331?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15318950#comment-15318950
]
Michael Wall commented on ACCUMULO-4331:
Looking for the code now that sets up the log4j file, but could you do
-Dproc=tserver -Dport=9997 and have affect the log file name?
> Make port configuration and allocation consistent across services
> -
>
> Key: ACCUMULO-4331
> URL: https://issues.apache.org/jira/browse/ACCUMULO-4331
> Project: Accumulo
> Issue Type: Bug
>Affects Versions: 1.8.0
>Reporter: Dave Marion
> Fix For: 1.8.0
>
>
> There was some discussion in ACCUMULO-4328 about ports, so I decided to track
> down how the client ports are configured and allocated. Issues raised in the
> discussion were:
> 1. The port search feature was not well understood
> 2. Ephemeral port allocation makes it hard to lock servers down (e.g.
> iptables)
> Looking through the code I found the following properties allocate a port
> number based on conf.getPort(). This returns the port number based on the
> property and supports either a single value or zero. Then, in the server
> component (monitor, tracer, gc, etc) this value is used when creating a
> ServerSocket. If the port is already in use, the process will fail.
> {noformat}
> monitor.port.log4j
> trace.port.client
> gc.port.client
> monitor.port.client
> {noformat}
> The following properties use TServerUtils.startServer which uses the value in
> the property to start the TServer. If the value is zero, then it picks a
> random port between 1024 and 65535. If tserver.port.search is enabled, then
> it will try a thousand times to bind to a random port.
> {noformat}
> tserver.port.client
> master.port.client
> master.replication.coordinator.port
> replication.receipt.service.port
> {noformat}
> I'm proposing that we deprecate the tserver.port.search property and the
> value zero in the property value for the properties above. Instead, I think
> we should allow the user to specify a single value or a range (M-N).
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (ACCUMULO-4331) Make port configuration and allocation consistent across services
[
https://issues.apache.org/jira/browse/ACCUMULO-4331?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15318917#comment-15318917
]
Josh Elser commented on ACCUMULO-4331:
--
bq. so you know automatically which log file to open instead of having to grep
for something in all the log files.
I know that Keith/Dave have been talking on the PR for ACCUMULO-4328, but maybe
something could be added (like our {{-Dproc=tserver}}) now that could help
correlate instances to log file. I feel like trying to include port will just
be tricky (because we want logging set up before we're going to bind the server
to a port).
> Make port configuration and allocation consistent across services
> -
>
> Key: ACCUMULO-4331
> URL: https://issues.apache.org/jira/browse/ACCUMULO-4331
> Project: Accumulo
> Issue Type: Bug
>Affects Versions: 1.8.0
>Reporter: Dave Marion
> Fix For: 1.8.0
>
>
> There was some discussion in ACCUMULO-4328 about ports, so I decided to track
> down how the client ports are configured and allocated. Issues raised in the
> discussion were:
> 1. The port search feature was not well understood
> 2. Ephemeral port allocation makes it hard to lock servers down (e.g.
> iptables)
> Looking through the code I found the following properties allocate a port
> number based on conf.getPort(). This returns the port number based on the
> property and supports either a single value or zero. Then, in the server
> component (monitor, tracer, gc, etc) this value is used when creating a
> ServerSocket. If the port is already in use, the process will fail.
> {noformat}
> monitor.port.log4j
> trace.port.client
> gc.port.client
> monitor.port.client
> {noformat}
> The following properties use TServerUtils.startServer which uses the value in
> the property to start the TServer. If the value is zero, then it picks a
> random port between 1024 and 65535. If tserver.port.search is enabled, then
> it will try a thousand times to bind to a random port.
> {noformat}
> tserver.port.client
> master.port.client
> master.replication.coordinator.port
> replication.receipt.service.port
> {noformat}
> I'm proposing that we deprecate the tserver.port.search property and the
> value zero in the property value for the properties above. Instead, I think
> we should allow the user to specify a single value or a range (M-N).
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (ACCUMULO-4331) Make port configuration and allocation consistent across services
[
https://issues.apache.org/jira/browse/ACCUMULO-4331?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15318909#comment-15318909
]
Michael Wall commented on ACCUMULO-4331:
It would be really useful if we are going to run multiple tservers on a node to
be able to set static ports. So if I want to run 3 on a node, I can configure
them to always run on 9997, 9998 and or whatever. That is useful from a
system monitoring perspective. Also useful because in the tserver page in the
monitor, when a tserver dies and comes back up on the same port, it would be
removed from the list of dead tservers. I find it useful to follow restarts of
a tserver in one log file as well, although I think [~dlmarion] has that
working already. Would love to see the port as part of the log4j filename as
well, so you know automatically which log file to open instead of having to
grep for something in all the log files.
> Make port configuration and allocation consistent across services
> -
>
> Key: ACCUMULO-4331
> URL: https://issues.apache.org/jira/browse/ACCUMULO-4331
> Project: Accumulo
> Issue Type: Bug
>Affects Versions: 1.8.0
>Reporter: Dave Marion
> Fix For: 1.8.0
>
>
> There was some discussion in ACCUMULO-4328 about ports, so I decided to track
> down how the client ports are configured and allocated. Issues raised in the
> discussion were:
> 1. The port search feature was not well understood
> 2. Ephemeral port allocation makes it hard to lock servers down (e.g.
> iptables)
> Looking through the code I found the following properties allocate a port
> number based on conf.getPort(). This returns the port number based on the
> property and supports either a single value or zero. Then, in the server
> component (monitor, tracer, gc, etc) this value is used when creating a
> ServerSocket. If the port is already in use, the process will fail.
> {noformat}
> monitor.port.log4j
> trace.port.client
> gc.port.client
> monitor.port.client
> {noformat}
> The following properties use TServerUtils.startServer which uses the value in
> the property to start the TServer. If the value is zero, then it picks a
> random port between 1024 and 65535. If tserver.port.search is enabled, then
> it will try a thousand times to bind to a random port.
> {noformat}
> tserver.port.client
> master.port.client
> master.replication.coordinator.port
> replication.receipt.service.port
> {noformat}
> I'm proposing that we deprecate the tserver.port.search property and the
> value zero in the property value for the properties above. Instead, I think
> we should allow the user to specify a single value or a range (M-N).
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (ACCUMULO-4328) Run multiple tablet servers on a single host
[ https://issues.apache.org/jira/browse/ACCUMULO-4328?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15318905#comment-15318905 ] Josh Elser commented on ACCUMULO-4328: -- bq. PR against 1.8 has been opened. Thanks, [~dlmarion]. I appreciate you switching the target version. I know it put more work on you. > Run multiple tablet servers on a single host > > > Key: ACCUMULO-4328 > URL: https://issues.apache.org/jira/browse/ACCUMULO-4328 > Project: Accumulo > Issue Type: Improvement > Components: scripts, tserver >Reporter: Dave Marion >Assignee: Dave Marion > Time Spent: 4h 40m > Remaining Estimate: 0h > > Modify scripts and necessary code to run multiple tablet servers on a single > host. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (ACCUMULO-4331) Make port configuration and allocation consistent across services
[
https://issues.apache.org/jira/browse/ACCUMULO-4331?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15318900#comment-15318900
]
Dave Marion commented on ACCUMULO-4331:
---
It makes it hard to lock ports down on a server (iptables)
> Make port configuration and allocation consistent across services
> -
>
> Key: ACCUMULO-4331
> URL: https://issues.apache.org/jira/browse/ACCUMULO-4331
> Project: Accumulo
> Issue Type: Bug
>Affects Versions: 1.8.0
>Reporter: Dave Marion
> Fix For: 1.8.0
>
>
> There was some discussion in ACCUMULO-4328 about ports, so I decided to track
> down how the client ports are configured and allocated. Issues raised in the
> discussion were:
> 1. The port search feature was not well understood
> 2. Ephemeral port allocation makes it hard to lock servers down (e.g.
> iptables)
> Looking through the code I found the following properties allocate a port
> number based on conf.getPort(). This returns the port number based on the
> property and supports either a single value or zero. Then, in the server
> component (monitor, tracer, gc, etc) this value is used when creating a
> ServerSocket. If the port is already in use, the process will fail.
> {noformat}
> monitor.port.log4j
> trace.port.client
> gc.port.client
> monitor.port.client
> {noformat}
> The following properties use TServerUtils.startServer which uses the value in
> the property to start the TServer. If the value is zero, then it picks a
> random port between 1024 and 65535. If tserver.port.search is enabled, then
> it will try a thousand times to bind to a random port.
> {noformat}
> tserver.port.client
> master.port.client
> master.replication.coordinator.port
> replication.receipt.service.port
> {noformat}
> I'm proposing that we deprecate the tserver.port.search property and the
> value zero in the property value for the properties above. Instead, I think
> we should allow the user to specify a single value or a range (M-N).
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (ACCUMULO-4331) Make port configuration and allocation consistent across services
[
https://issues.apache.org/jira/browse/ACCUMULO-4331?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15318893#comment-15318893
]
Josh Elser commented on ACCUMULO-4331:
--
bq. I'd prefer not deprecating the 0 option. Accumulo on Slider uses that.
Yeah, It doesn't make sense to me to deprecate using random ports, it's just
not something that Accumulo out of the box would recommend (because of what we
know about production environments of security-conscious people). I think that
adequate documentation about when '0' makes sense to use would be better than
removing entirely.
bq. Instead, I think we should allow the user to specify a single value or a
range (M-N).
I think this would be a nice thing to add.
> Make port configuration and allocation consistent across services
> -
>
> Key: ACCUMULO-4331
> URL: https://issues.apache.org/jira/browse/ACCUMULO-4331
> Project: Accumulo
> Issue Type: Bug
>Affects Versions: 1.8.0
>Reporter: Dave Marion
> Fix For: 1.8.0
>
>
> There was some discussion in ACCUMULO-4328 about ports, so I decided to track
> down how the client ports are configured and allocated. Issues raised in the
> discussion were:
> 1. The port search feature was not well understood
> 2. Ephemeral port allocation makes it hard to lock servers down (e.g.
> iptables)
> Looking through the code I found the following properties allocate a port
> number based on conf.getPort(). This returns the port number based on the
> property and supports either a single value or zero. Then, in the server
> component (monitor, tracer, gc, etc) this value is used when creating a
> ServerSocket. If the port is already in use, the process will fail.
> {noformat}
> monitor.port.log4j
> trace.port.client
> gc.port.client
> monitor.port.client
> {noformat}
> The following properties use TServerUtils.startServer which uses the value in
> the property to start the TServer. If the value is zero, then it picks a
> random port between 1024 and 65535. If tserver.port.search is enabled, then
> it will try a thousand times to bind to a random port.
> {noformat}
> tserver.port.client
> master.port.client
> master.replication.coordinator.port
> replication.receipt.service.port
> {noformat}
> I'm proposing that we deprecate the tserver.port.search property and the
> value zero in the property value for the properties above. Instead, I think
> we should allow the user to specify a single value or a range (M-N).
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (ACCUMULO-4331) Make port configuration and allocation consistent across services
[
https://issues.apache.org/jira/browse/ACCUMULO-4331?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15318876#comment-15318876
]
Billie Rinaldi commented on ACCUMULO-4331:
--
Not really; it's easier to set it to 0 when you don't care what the port is,
and that's consistent with other projects (e.g. HBase). What's the argument for
not allowing 0?
> Make port configuration and allocation consistent across services
> -
>
> Key: ACCUMULO-4331
> URL: https://issues.apache.org/jira/browse/ACCUMULO-4331
> Project: Accumulo
> Issue Type: Bug
>Affects Versions: 1.8.0
>Reporter: Dave Marion
> Fix For: 1.8.0
>
>
> There was some discussion in ACCUMULO-4328 about ports, so I decided to track
> down how the client ports are configured and allocated. Issues raised in the
> discussion were:
> 1. The port search feature was not well understood
> 2. Ephemeral port allocation makes it hard to lock servers down (e.g.
> iptables)
> Looking through the code I found the following properties allocate a port
> number based on conf.getPort(). This returns the port number based on the
> property and supports either a single value or zero. Then, in the server
> component (monitor, tracer, gc, etc) this value is used when creating a
> ServerSocket. If the port is already in use, the process will fail.
> {noformat}
> monitor.port.log4j
> trace.port.client
> gc.port.client
> monitor.port.client
> {noformat}
> The following properties use TServerUtils.startServer which uses the value in
> the property to start the TServer. If the value is zero, then it picks a
> random port between 1024 and 65535. If tserver.port.search is enabled, then
> it will try a thousand times to bind to a random port.
> {noformat}
> tserver.port.client
> master.port.client
> master.replication.coordinator.port
> replication.receipt.service.port
> {noformat}
> I'm proposing that we deprecate the tserver.port.search property and the
> value zero in the property value for the properties above. Instead, I think
> we should allow the user to specify a single value or a range (M-N).
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (ACCUMULO-4331) Make port configuration and allocation consistent across services
[
https://issues.apache.org/jira/browse/ACCUMULO-4331?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15318830#comment-15318830
]
Dave Marion commented on ACCUMULO-4331:
---
It couldn't be removed until 2.0. Does that change things?
> Make port configuration and allocation consistent across services
> -
>
> Key: ACCUMULO-4331
> URL: https://issues.apache.org/jira/browse/ACCUMULO-4331
> Project: Accumulo
> Issue Type: Bug
>Affects Versions: 1.8.0
>Reporter: Dave Marion
> Fix For: 1.8.0
>
>
> There was some discussion in ACCUMULO-4328 about ports, so I decided to track
> down how the client ports are configured and allocated. Issues raised in the
> discussion were:
> 1. The port search feature was not well understood
> 2. Ephemeral port allocation makes it hard to lock servers down (e.g.
> iptables)
> Looking through the code I found the following properties allocate a port
> number based on conf.getPort(). This returns the port number based on the
> property and supports either a single value or zero. Then, in the server
> component (monitor, tracer, gc, etc) this value is used when creating a
> ServerSocket. If the port is already in use, the process will fail.
> {noformat}
> monitor.port.log4j
> trace.port.client
> gc.port.client
> monitor.port.client
> {noformat}
> The following properties use TServerUtils.startServer which uses the value in
> the property to start the TServer. If the value is zero, then it picks a
> random port between 1024 and 65535. If tserver.port.search is enabled, then
> it will try a thousand times to bind to a random port.
> {noformat}
> tserver.port.client
> master.port.client
> master.replication.coordinator.port
> replication.receipt.service.port
> {noformat}
> I'm proposing that we deprecate the tserver.port.search property and the
> value zero in the property value for the properties above. Instead, I think
> we should allow the user to specify a single value or a range (M-N).
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (ACCUMULO-4331) Make port configuration and allocation consistent across services
[
https://issues.apache.org/jira/browse/ACCUMULO-4331?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15318827#comment-15318827
]
Billie Rinaldi commented on ACCUMULO-4331:
--
I'd prefer not deprecating the 0 option. Accumulo on Slider uses that.
> Make port configuration and allocation consistent across services
> -
>
> Key: ACCUMULO-4331
> URL: https://issues.apache.org/jira/browse/ACCUMULO-4331
> Project: Accumulo
> Issue Type: Bug
>Affects Versions: 1.8.0
>Reporter: Dave Marion
> Fix For: 1.8.0
>
>
> There was some discussion in ACCUMULO-4328 about ports, so I decided to track
> down how the client ports are configured and allocated. Issues raised in the
> discussion were:
> 1. The port search feature was not well understood
> 2. Ephemeral port allocation makes it hard to lock servers down (e.g.
> iptables)
> Looking through the code I found the following properties allocate a port
> number based on conf.getPort(). This returns the port number based on the
> property and supports either a single value or zero. Then, in the server
> component (monitor, tracer, gc, etc) this value is used when creating a
> ServerSocket. If the port is already in use, the process will fail.
> {noformat}
> monitor.port.log4j
> trace.port.client
> gc.port.client
> monitor.port.client
> {noformat}
> The following properties use TServerUtils.startServer which uses the value in
> the property to start the TServer. If the value is zero, then it picks a
> random port between 1024 and 65535. If tserver.port.search is enabled, then
> it will try a thousand times to bind to a random port.
> {noformat}
> tserver.port.client
> master.port.client
> master.replication.coordinator.port
> replication.receipt.service.port
> {noformat}
> I'm proposing that we deprecate the tserver.port.search property and the
> value zero in the property value for the properties above. Instead, I think
> we should allow the user to specify a single value or a range (M-N).
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
Accumulo-Pull-Requests - Build # 322 - Failure
The Apache Jenkins build system has built Accumulo-Pull-Requests (build #322) Status: Failure Check console output at https://builds.apache.org/job/Accumulo-Pull-Requests/322/ to view the results.
[jira] [Created] (ACCUMULO-4332) Tserver exiting due to uncaught error
Michael Wall created ACCUMULO-4332:
--
Summary: Tserver exiting due to uncaught error
Key: ACCUMULO-4332
URL: https://issues.apache.org/jira/browse/ACCUMULO-4332
Project: Accumulo
Issue Type: Bug
Affects Versions: 1.6.5
Reporter: Michael Wall
After the following log statements, the tserver became a zombie and did nothing
but spit GC and Cache log statements. The fix was to add more open files,
ulimit -n was 1024. This ticket is to see if we can catch this exception and
kill the tserver.
{code}
ERROR: run() exiting due to uncaught error
java.lang.OutOfMemoryError: unable to create new native thread
at java.lang.Thread.start0(Native Method)
at java.lang.Thread.start(Thread.java:714)
at
java.util.concurrent.ThreadPoolExecutor.addWorker(ThreadPoolExecutor.java:949)
at
java.util.concurrent.ThreadPoolExecutor.execute(ThreadPoolExecutor.java:1360)
at org.apache.thrift.server.THsHaServer.requestInvoke(THsHaServer.java:157)
at
org.apache.thrift.server.AbstractNonblockingServer$AbstractSelectThread.handleRead(AbstractNonblockingServer.java:210)
at
org.apache.accumulo.server.util.CustomNonBlockerServer$SelectAcceptThread.select(CustomNonBlockingServer.java:227)
at
org.apache.accumulo.server.util.CustomNonBlockerServer$SelectAcceptThread.run(CustomNonBlockingServer.java:183)
{code}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (ACCUMULO-4330) ZooKeeper test logs way too big
[ https://issues.apache.org/jira/browse/ACCUMULO-4330?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Christopher Tubbs updated ACCUMULO-4330: Priority: Critical (was: Major) > ZooKeeper test logs way too big > --- > > Key: ACCUMULO-4330 > URL: https://issues.apache.org/jira/browse/ACCUMULO-4330 > Project: Accumulo > Issue Type: Bug > Components: test >Reporter: Christopher Tubbs >Priority: Critical > Fix For: 1.6.6, 1.7.2, 1.8.0 > > > Could not complete a run of the ITs on Jenkins because the > ZooKeeperServerMain_\*.out files were far too big. > The ZooKeeperServerMain log file for > org.apache.accumulo.harness.SharedMiniClusterBase grew to over 17G before > filling up my disk. > MiniAccumuloCluster probably should not write the DEBUG logs for ZooKeeper by > default. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (ACCUMULO-4330) ZooKeeper test logs way too big
[ https://issues.apache.org/jira/browse/ACCUMULO-4330?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15317985#comment-15317985 ] Christopher Tubbs commented on ACCUMULO-4330: - Bumping this up in priority only because it makes it harder to run the full tests for releases if they continue to take up so much disk space. > ZooKeeper test logs way too big > --- > > Key: ACCUMULO-4330 > URL: https://issues.apache.org/jira/browse/ACCUMULO-4330 > Project: Accumulo > Issue Type: Bug > Components: test >Reporter: Christopher Tubbs >Priority: Critical > Fix For: 1.6.6, 1.7.2, 1.8.0 > > > Could not complete a run of the ITs on Jenkins because the > ZooKeeperServerMain_\*.out files were far too big. > The ZooKeeperServerMain log file for > org.apache.accumulo.harness.SharedMiniClusterBase grew to over 17G before > filling up my disk. > MiniAccumuloCluster probably should not write the DEBUG logs for ZooKeeper by > default. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (ACCUMULO-4330) ZooKeeper test logs way too big
[ https://issues.apache.org/jira/browse/ACCUMULO-4330?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Christopher Tubbs updated ACCUMULO-4330: Fix Version/s: 1.8.0 1.7.2 1.6.6 > ZooKeeper test logs way too big > --- > > Key: ACCUMULO-4330 > URL: https://issues.apache.org/jira/browse/ACCUMULO-4330 > Project: Accumulo > Issue Type: Bug > Components: test >Reporter: Christopher Tubbs > Fix For: 1.6.6, 1.7.2, 1.8.0 > > > Could not complete a run of the ITs on Jenkins because the > ZooKeeperServerMain_\*.out files were far too big. > The ZooKeeperServerMain log file for > org.apache.accumulo.harness.SharedMiniClusterBase grew to over 17G before > filling up my disk. > MiniAccumuloCluster probably should not write the DEBUG logs for ZooKeeper by > default. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
