[jira] [Commented] (CASSANDRA-12437) dtest failure in bootstrap_test.TestBootstrap.local_quorum_bootstrap_test
[
https://issues.apache.org/jira/browse/CASSANDRA-12437?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15842964#comment-15842964
]
Philip Thompson commented on CASSANDRA-12437:
-
Why would we not just specify the correct port for stress to connect to?
> dtest failure in bootstrap_test.TestBootstrap.local_quorum_bootstrap_test
> -
>
> Key: CASSANDRA-12437
> URL: https://issues.apache.org/jira/browse/CASSANDRA-12437
> Project: Cassandra
> Issue Type: Test
>Reporter: Craig Kodman
>Assignee: DS Test Eng
> Labels: dtest, windows
> Attachments: node1_debug.log, node1_gc.log, node1.log,
> node2_debug.log, node2_gc.log, node2.log
>
>
> example failure:
> http://cassci.datastax.com/job/cassandra-2.2_dtest_win32/281/testReport/bootstrap_test/TestBootstrap/local_quorum_bootstrap_test
> {code}
> Stacktrace
> File "C:\tools\python2\lib\unittest\case.py", line 329, in run
> testMethod()
> File
> "D:\jenkins\workspace\cassandra-2.2_dtest_win32\cassandra-dtest\bootstrap_test.py",
> line 389, in local_quorum_bootstrap_test
> 'ops(insert=1)', '-rate', 'threads=50'])
> File "D:\jenkins\workspace\cassandra-2.2_dtest_win32\ccm\ccmlib\node.py",
> line 1244, in stress
> return handle_external_tool_process(p, ['stress'] + stress_options)
> File "D:\jenkins\workspace\cassandra-2.2_dtest_win32\ccm\ccmlib\node.py",
> line 1955, in handle_external_tool_process
> raise ToolError(cmd_args, rc, out, err)
> 'Subprocess [\'stress\', \'user\', \'profile=d:temp2tmp8sf4da\',
> \'n=2M\', \'no-warmup\', \'ops(insert=1)\', \'-rate\', \'threads=50\'] exited
> with non-zero status; exit status: 1; \nstderr: Exception in thread "main"
> java.io.IOError: java.io.FileNotFoundException: d:\\temp\\2\\tmp8sf4da (The
> process cannot access the file because it is being used by another
> process)\r\n\tat
> org.apache.cassandra.stress.StressProfile.load(StressProfile.java:574)\r\n\tat
>
> org.apache.cassandra.stress.settings.SettingsCommandUser.(SettingsCommandUser.java:58)\r\n\tat
>
> org.apache.cassandra.stress.settings.SettingsCommandUser.build(SettingsCommandUser.java:127)\r\n\tat
>
> org.apache.cassandra.stress.settings.SettingsCommand.get(SettingsCommand.java:195)\r\n\tat
>
> org.apache.cassandra.stress.settings.StressSettings.get(StressSettings.java:249)\r\n\tat
>
> org.apache.cassandra.stress.settings.StressSettings.parse(StressSettings.java:220)\r\n\tat
> org.apache.cassandra.stress.Stress.main(Stress.java:63)\r\nCaused by:
> java.io.FileNotFoundException: d:\\temp\\2\\tmp8sf4da (The process cannot
> access the file because it is being used by another process)\r\n\tat
> java.io.FileInputStream.open0(Native Method)\r\n\tat
> java.io.FileInputStream.open(FileInputStream.java:195)\r\n\tat
> java.io.FileInputStream.(FileInputStream.java:138)\r\n\tat
> java.io.FileInputStream.(FileInputStream.java:93)\r\n\tat
> sun.net.www.protocol.file.FileURLConnection.connect(FileURLConnection.java:90)\r\n\tat
>
> sun.net.www.protocol.file.FileURLConnection.getInputStream(FileURLConnection.java:188)\r\n\tat
> java.net.URL.openStream(URL.java:1038)\r\n\tat
> org.apache.cassandra.stress.StressProfile.load(StressProfile.java:560)\r\n\t...
> 6 more\r\n\n >> begin captured logging <<
> \ndtest: DEBUG: cluster ccm directory:
> d:\\temp\\2\\dtest-wsze0r\ndtest: DEBUG: Done setting configuration
> options:\n{ \'initial_token\': None,\n\'num_tokens\': \'32\',\n
> \'phi_convict_threshold\': 5,\n\'start_rpc\':
> \'true\'}\n- >> end captured logging <<
> -'
> {code}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (CASSANDRA-12437) dtest failure in bootstrap_test.TestBootstrap.local_quorum_bootstrap_test
[
https://issues.apache.org/jira/browse/CASSANDRA-12437?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15842443#comment-15842443
]
Stefan Podkowinski commented on CASSANDRA-12437:
Looks like stress is trying to connect using the default 7199 port, while we
start with 7100 for the first node when using ccm.
As this is broken for quite a while and no other tests seem to be affected, I'd
assume it's okay to simply add the error to the list of ignore patterns unless
we actual need the additional stress output.
[PR|https://github.com/riptano/cassandra-dtest/pull/1429] has been created.
/cc [~philipthompson]
> dtest failure in bootstrap_test.TestBootstrap.local_quorum_bootstrap_test
> -
>
> Key: CASSANDRA-12437
> URL: https://issues.apache.org/jira/browse/CASSANDRA-12437
> Project: Cassandra
> Issue Type: Test
>Reporter: Craig Kodman
>Assignee: DS Test Eng
> Labels: dtest, windows
> Attachments: node1_debug.log, node1_gc.log, node1.log,
> node2_debug.log, node2_gc.log, node2.log
>
>
> example failure:
> http://cassci.datastax.com/job/cassandra-2.2_dtest_win32/281/testReport/bootstrap_test/TestBootstrap/local_quorum_bootstrap_test
> {code}
> Stacktrace
> File "C:\tools\python2\lib\unittest\case.py", line 329, in run
> testMethod()
> File
> "D:\jenkins\workspace\cassandra-2.2_dtest_win32\cassandra-dtest\bootstrap_test.py",
> line 389, in local_quorum_bootstrap_test
> 'ops(insert=1)', '-rate', 'threads=50'])
> File "D:\jenkins\workspace\cassandra-2.2_dtest_win32\ccm\ccmlib\node.py",
> line 1244, in stress
> return handle_external_tool_process(p, ['stress'] + stress_options)
> File "D:\jenkins\workspace\cassandra-2.2_dtest_win32\ccm\ccmlib\node.py",
> line 1955, in handle_external_tool_process
> raise ToolError(cmd_args, rc, out, err)
> 'Subprocess [\'stress\', \'user\', \'profile=d:temp2tmp8sf4da\',
> \'n=2M\', \'no-warmup\', \'ops(insert=1)\', \'-rate\', \'threads=50\'] exited
> with non-zero status; exit status: 1; \nstderr: Exception in thread "main"
> java.io.IOError: java.io.FileNotFoundException: d:\\temp\\2\\tmp8sf4da (The
> process cannot access the file because it is being used by another
> process)\r\n\tat
> org.apache.cassandra.stress.StressProfile.load(StressProfile.java:574)\r\n\tat
>
> org.apache.cassandra.stress.settings.SettingsCommandUser.(SettingsCommandUser.java:58)\r\n\tat
>
> org.apache.cassandra.stress.settings.SettingsCommandUser.build(SettingsCommandUser.java:127)\r\n\tat
>
> org.apache.cassandra.stress.settings.SettingsCommand.get(SettingsCommand.java:195)\r\n\tat
>
> org.apache.cassandra.stress.settings.StressSettings.get(StressSettings.java:249)\r\n\tat
>
> org.apache.cassandra.stress.settings.StressSettings.parse(StressSettings.java:220)\r\n\tat
> org.apache.cassandra.stress.Stress.main(Stress.java:63)\r\nCaused by:
> java.io.FileNotFoundException: d:\\temp\\2\\tmp8sf4da (The process cannot
> access the file because it is being used by another process)\r\n\tat
> java.io.FileInputStream.open0(Native Method)\r\n\tat
> java.io.FileInputStream.open(FileInputStream.java:195)\r\n\tat
> java.io.FileInputStream.(FileInputStream.java:138)\r\n\tat
> java.io.FileInputStream.(FileInputStream.java:93)\r\n\tat
> sun.net.www.protocol.file.FileURLConnection.connect(FileURLConnection.java:90)\r\n\tat
>
> sun.net.www.protocol.file.FileURLConnection.getInputStream(FileURLConnection.java:188)\r\n\tat
> java.net.URL.openStream(URL.java:1038)\r\n\tat
> org.apache.cassandra.stress.StressProfile.load(StressProfile.java:560)\r\n\t...
> 6 more\r\n\n >> begin captured logging <<
> \ndtest: DEBUG: cluster ccm directory:
> d:\\temp\\2\\dtest-wsze0r\ndtest: DEBUG: Done setting configuration
> options:\n{ \'initial_token\': None,\n\'num_tokens\': \'32\',\n
> \'phi_convict_threshold\': 5,\n\'start_rpc\':
> \'true\'}\n- >> end captured logging <<
> -'
> {code}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (CASSANDRA-12437) dtest failure in bootstrap_test.TestBootstrap.local_quorum_bootstrap_test
[
https://issues.apache.org/jira/browse/CASSANDRA-12437?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15454844#comment-15454844
]
Sylvain Lebresne commented on CASSANDRA-12437:
--
I just go a failure of this test on a [dev
run|http://cassci.datastax.com/job/pcmanus-12527-3.0-dtest/lastBuild/testReport/junit/bootstrap_test/TestBootstrap/local_quorum_bootstrap_test/]
that failed with the following problem:
{noformat}
Attempted to check that stderr was empty. Instead, stderr is Failed to connect
over JMX; not collecting these stats
, but the regex used to check against stderr is ^(WARN.*JNA link
failure.*unavailable.|objc.*Class JavaLaunchHelper.*?Which one is
undefined.|\s*|
)*$
>> begin captured logging <<
dtest: DEBUG: cluster ccm directory: /tmp/dtest-OcCcHm
dtest: DEBUG: Done setting configuration options:
{ 'initial_token': None,
'num_tokens': '32',
'phi_convict_threshold': 5,
'start_rpc': 'true'}
dtest: DEBUG: Connected to cluster: test, max pending requests per connection
128, max connections per host 8
Datatacenter: dc1; Host: /127.0.0.1; Rack: r1
Datatacenter: dc2; Host: /127.0.0.2; Rack: r1
Created schema. Sleeping 1s for propagation.
Sleeping 2s...
Running [insert] with 5 threads for 50 iteration
type, total ops,op/s,pk/s, row/s,mean, med, .95,
.99,.999, max, time, stderr, errors, gc: #, max ms, sum ms, sdv
ms, mb
Generating batches with [1..1] partitions and [1..1] rows (of [1..1] total rows
in the partitions)
total, 1086,1086,1086,1086, 4.5, 3.3,10.9,
25.1,41.8,51.3,1.0, 0.0, 0, 0, 0, 0,
0, 0
total, 2896,1751,1751,1751, 2.8, 2.1, 6.4,
17.3,42.5,43.0,2.0, 0.17929, 0, 0, 0, 0,
0, 0
total, 5342,2433,2433,2433, 2.0, 1.7, 4.1,
7.3,15.0,21.0,3.0, 0.17863, 0, 0, 0, 0,
0, 0
{noformat}
where the error on stderr is something that stress does write when it can't
connect on JMX. It's not the same error than in the description but both error
sound like there are an environment issues and not genuine bug, so figured
whomever will mess with this ticket could try to kill two birds with one stone
rather than created tons of tickets.
I'll note that I don't know why stress can't connect through JMX while it's
able to run successfully otherwise. If that happened all the time I'd suggest
an authentication failure but as I seem to be the first to report this, I
assume it's flapping and suggest maybe some race in JMX initialization? In any
case, this doesn't prevent stress from running and that all that matters for
the test, so a simple fix to avoid this problem could be to ignore this error
message entirely, as it's really not a problem for the test.
> dtest failure in bootstrap_test.TestBootstrap.local_quorum_bootstrap_test
> -
>
> Key: CASSANDRA-12437
> URL: https://issues.apache.org/jira/browse/CASSANDRA-12437
> Project: Cassandra
> Issue Type: Test
>Reporter: Craig Kodman
>Assignee: DS Test Eng
> Labels: dtest, windows
> Attachments: node1.log, node1_debug.log, node1_gc.log, node2.log,
> node2_debug.log, node2_gc.log
>
>
> example failure:
> http://cassci.datastax.com/job/cassandra-2.2_dtest_win32/281/testReport/bootstrap_test/TestBootstrap/local_quorum_bootstrap_test
> {code}
> Stacktrace
> File "C:\tools\python2\lib\unittest\case.py", line 329, in run
> testMethod()
> File
> "D:\jenkins\workspace\cassandra-2.2_dtest_win32\cassandra-dtest\bootstrap_test.py",
> line 389, in local_quorum_bootstrap_test
> 'ops(insert=1)', '-rate', 'threads=50'])
> File "D:\jenkins\workspace\cassandra-2.2_dtest_win32\ccm\ccmlib\node.py",
> line 1244, in stress
> return handle_external_tool_process(p, ['stress'] + stress_options)
> File "D:\jenkins\workspace\cassandra-2.2_dtest_win32\ccm\ccmlib\node.py",
> line 1955, in handle_external_tool_process
> raise ToolError(cmd_args, rc, out, err)
> 'Subprocess [\'stress\', \'user\', \'profile=d:temp2tmp8sf4da\',
> \'n=2M\', \'no-warmup\', \'ops(insert=1)\', \'-rate\', \'threads=50\'] exited
> with non-zero status; exit status: 1; \nstderr: Exception in thread "main"
> java.io.IOError: java.io.FileNotFoundException: d:\\temp\\2\\tmp8sf4da (The
> process cannot access the file because it is being used by another
> process)\r\n\tat
> org.apache.cassandra.stress.StressProfile.load(StressProfile.java:574)\r\n\tat
>
> org.apache.cassandra.stress.settings.SettingsCommandUser.(SettingsCommandUser.jav
