Re: Review Request 65771: Add .travis.yml
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/65771/#review198341 --- Ship it! Ship It! - Szabolcs Vasas On Feb. 23, 2018, 1:48 p.m., daniel voros wrote: > > --- > This is an automatically generated e-mail. To reply, visit: > https://reviews.apache.org/r/65771/ > --- > > (Updated Feb. 23, 2018, 1:48 p.m.) > > > Review request for Sqoop. > > > Bugs: SQOOP-3289 > https://issues.apache.org/jira/browse/SQOOP-3289 > > > Repository: sqoop-trunk > > > Description > --- > > Adding a .travis.yml would enable running builds/tests on travis-ci.org. > Currently if you wish to use Travis for testing your changes, you have to > manually add a .travis.yml to your branch. Having it committed to trunk would > save us this extra step. > > > Diffs > - > > .travis.yml PRE-CREATION > > > Diff: https://reviews.apache.org/r/65771/diff/1/ > > > Testing > --- > > See https://travis-ci.org/dvoros/sqoop/builds > > > Thanks, > > daniel voros > >
[jira] [Commented] (SQOOP-3289) Add .travis.yml
[
https://issues.apache.org/jira/browse/SQOOP-3289?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16379181#comment-16379181
]
Szabolcs Vasas commented on SQOOP-3289:
---
Hi [~dvoros],
Thank you for raising this JIRA and bringing up this topic!
I have also experimented with Travis recently it's a great chance to share our
thoughts on it.
TL;DR: +1 for the patch, I think it can make the life of the contributors
easier and it is a good start for further improvements.
I think the benefit of Travis over Jenkins is that the contributors can run it
on their own forks and can get very quick feedback (the dependency caching
minimizes the time spent with the ivy resolve).
I understand your patch addresses this use case only but we can talk about how
we want/could extend it in the future.
I like that Travis integrates well with Github and pull requests so if we want
to use pull requests instead of review board enabling Travis on Sqoop trunk
could simplify the review process. I also find it great that .travis.yml is
part of the project so it is much easier to change it and it is more
transparent what is happening in the CI job. We might be able to do the same
with Jenkins but I am not really familiar with it and the permission
restrictions make it much harder to configure it.
One potential issue I can see is that Travis has a 50 minute timeout for the
jobs (https://docs.travis-ci.com/user/customizing-the-build/#Build-Timeouts).
However the good thing is that we could parallelize our build and have separate
jobs for our unit tests, integration tests, MySQL tests, etc. under the same
build
(https://docs.travis-ci.com/user/speeding-up-the-build/#Parallelizing-your-builds-across-virtual-machines).
This requires having a much better test category definition
(https://issues.apache.org/jira/browse/SQOOP-3104) but it seems to be feasible
to cover most of our test cases.
Another benefit is that Travis supports starting Docker containers in the build
(https://docs.travis-ci.com/user/docker) this could help us setup our
Docker-based third party test infrastructure much easier.
I would like to continue experimenting with Travis, please let me know what you
think about the above and as always all the contributions are welcome.
Szabolcs
> Add .travis.yml
> ---
>
> Key: SQOOP-3289
> URL: https://issues.apache.org/jira/browse/SQOOP-3289
> Project: Sqoop
> Issue Type: Task
> Components: build
>Affects Versions: 1.4.7
>Reporter: Daniel Voros
>Assignee: Daniel Voros
>Priority: Minor
> Fix For: 1.5.0
>
>
> Adding a .travis.yml would enable running builds/tests on travis-ci.org.
> Currently if you wish to use Travis for testing your changes, you have to
> manually add a .travis.yml to your branch. Having it committed to trunk would
> save us this extra step.
> I currently have an example
> [{{.travis.yml}}|https://github.com/dvoros/sqoop/blob/93a4c06c1a3da1fd5305c99e379484507797b3eb/.travis.yml]
> on my travis branch running unit tests for every commit and every pull
> request: https://travis-ci.org/dvoros/sqoop/builds
> Later we could add the build status to the project readme as well, see:
> https://github.com/dvoros/sqoop/tree/travis
> Also, an example of a pull request: https://github.com/dvoros/sqoop/pull/1
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (SQOOP-3042) Sqoop does not clear compile directory under /tmp/sqoop-/compile automatically
[
https://issues.apache.org/jira/browse/SQOOP-3042?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16378726#comment-16378726
]
Daniel Kozlowski commented on SQOOP-3042:
-
[~ericlin]
What sqoop version will this fix be embedded into?
> Sqoop does not clear compile directory under /tmp/sqoop-/compile
> automatically
>
>
> Key: SQOOP-3042
> URL: https://issues.apache.org/jira/browse/SQOOP-3042
> Project: Sqoop
> Issue Type: Bug
>Affects Versions: 1.4.6
>Reporter: Eric Lin
>Assignee: Eric Lin
>Priority: Critical
> Labels: patch
> Attachments: SQOOP-3042.1.patch, SQOOP-3042.2.patch,
> SQOOP-3042.4.patch
>
>
> After running sqoop, all the temp files generated by ClassWriter are left
> behind on disk, so anyone can check those JAVA files to see the schema of
> those tables that Sqoop has been interacting with. By default, the directory
> is under /tmp/sqoop-/compile.
> In class org.apache.sqoop.SqoopOptions, function getNonceJarDir(), I can see
> that we did add "deleteOnExit" on the temp dir:
> {code}
> for (int attempts = 0; attempts < MAX_DIR_CREATE_ATTEMPTS; attempts++) {
> hashDir = new File(baseDir, RandomHash.generateMD5String());
> while (hashDir.exists()) {
> hashDir = new File(baseDir, RandomHash.generateMD5String());
> }
> if (hashDir.mkdirs()) {
> // We created the directory. Use it.
> // If this directory is not actually filled with files, delete it
> // when the JVM quits.
> hashDir.deleteOnExit();
> break;
> }
> }
> {code}
> However, I believe it failed to delete due to directory is not empty.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
