[
https://issues.apache.org/jira/browse/SAMZA-508?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Chris Riccomini updated SAMZA-508:
----------------------------------
Affects Version/s: 0.9.0
> Supporting integration test execution on remote machines
> --------------------------------------------------------
>
> Key: SAMZA-508
> URL: https://issues.apache.org/jira/browse/SAMZA-508
> Project: Samza
> Issue Type: Bug
> Components: test
> Affects Versions: 0.9.0
> Reporter: Chris Riccomini
>
> SAMZA-468 added support for single-node 'localhost' execution of integration
> tests (YARN, Kafka, ZK, and Samza jobs are all deployed locally). Deploying
> tests to a single non-local host machine don't work right now because of the
> way we're deploying job tarballs. We're using yarn.package.path=file:/... URI
> schemes, and SCP'ing the job tarballs to all machines. This is a problem when
> executing on a non-local node because YARN stats the local job tarball when
> run-job.sh is run, and compares the tarball's
> [ctime/mtime/atime|http://www.linux-faqs.info/general/difference-between-mtime-ctime-and-atime]
> fields against the remote machines. They don't match since we're SCP'ing
> them, so YARN fails the job's deployment.
> We'll have to either modify the remote timestamps, or use some other
> filesystem (HDFS, HTTP, etc) to deploy the job tarballs to YARN NMs.
> This bug can be triggered simply by replacing all instances of `localhost` in
> samza-test/src/main/python/config/*json with a different machine's hostname.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
