[jira] [Commented] (IMPALA-7698) Add centos/redhat 6/7 support to bootstrap_system.sh
[
https://issues.apache.org/jira/browse/IMPALA-7698?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16751381#comment-16751381
]
ASF subversion and git services commented on IMPALA-7698:
-
Commit 81d0bcb3c967bf76b6858b221416e6fcb863b187 in impala's branch
refs/heads/master from Philip Zeyliger
[ https://gitbox.apache.org/repos/asf?p=impala.git;h=81d0bcb ]
Support centos:7 for test-with-docker.
As a follow-on to IMPALA-7698, adds various incantations
so that centos:7 can build under test-with-docker.
The core issue is that the centos:7 image doesn't let you start sshd
(necessary for the HBase startup scripts, and probably could be worked
around) or postgresql (harder to work around) with systemctl, because
systemd isn't "running." To avoid this, we start them manually
with /usr/sbin/sshd and pg_ctl.
Change-Id: I7577949b6eaaa2239bcf0fadf64e1490c2106b08
Reviewed-on: http://gerrit.cloudera.org:8080/12139
Reviewed-by: Impala Public Jenkins
Tested-by: Impala Public Jenkins
> Add centos/redhat 6/7 support to bootstrap_system.sh
>
>
> Key: IMPALA-7698
> URL: https://issues.apache.org/jira/browse/IMPALA-7698
> Project: IMPALA
> Issue Type: Task
> Components: Infrastructure
>Reporter: Philip Zeyliger
>Assignee: Philip Zeyliger
>Priority: Major
>
> {{bootstrap_system.sh}} currently only works on Ubuntu. Making it work on
> CentOS/Redhat would open the door to running automated tests on those
> platforms more readily, including using {{test-with-docker}}.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
-
To unsubscribe, e-mail: issues-all-unsubscr...@impala.apache.org
For additional commands, e-mail: issues-all-h...@impala.apache.org
[jira] [Commented] (IMPALA-7698) Add centos/redhat 6/7 support to bootstrap_system.sh
[
https://issues.apache.org/jira/browse/IMPALA-7698?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16745668#comment-16745668
]
ASF subversion and git services commented on IMPALA-7698:
-
Commit 0771e23e0b11b7a78256672a54e95b21dea05f0a in impala's branch
refs/heads/master from Philip Zeyliger
[ https://gitbox.apache.org/repos/asf?p=impala.git;h=0771e23 ]
Fix centos/Java/ORC default timezones in test-with-docker.
Stops configuring /etc/timezone for CentOS machines, which don't
typically have this file. Uses a longer format ("tz database name") for
/etc/timezone for Ubuntu, since that's what Ubuntu seems to expect. The
existing approach seemed to work, but it seems more consistent to use
the tz name.
To debug this, I wrote the following Java program:
import java.util.TimeZone;
public class test {
public static void main(String[] args) {
System.out.println(TimeZone.getDefault());
}
}
Running it under strace, with the OpenJDK source open to
src/solaris/native/java/util/TimeZone_md.c, I was able to
spot the issue. My previous attempt (IMPALA-7698,
c1701074d6e94d98a43ab049ef807ac1b368180f)
tread down this same path, but I had missed the failure.
Change-Id: I5dd7d823189e00edae4249d436bedfe4dd05a3a1
Reviewed-on: http://gerrit.cloudera.org:8080/12137
Reviewed-by: Impala Public Jenkins
Reviewed-by: Laszlo Gaal
Tested-by: Impala Public Jenkins
> Add centos/redhat 6/7 support to bootstrap_system.sh
>
>
> Key: IMPALA-7698
> URL: https://issues.apache.org/jira/browse/IMPALA-7698
> Project: IMPALA
> Issue Type: Task
> Components: Infrastructure
>Reporter: Philip Zeyliger
>Assignee: Philip Zeyliger
>Priority: Major
>
> {{bootstrap_system.sh}} currently only works on Ubuntu. Making it work on
> CentOS/Redhat would open the door to running automated tests on those
> platforms more readily, including using {{test-with-docker}}.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
-
To unsubscribe, e-mail: issues-all-unsubscr...@impala.apache.org
For additional commands, e-mail: issues-all-h...@impala.apache.org
[jira] [Commented] (IMPALA-7698) Add centos/redhat 6/7 support to bootstrap_system.sh
[
https://issues.apache.org/jira/browse/IMPALA-7698?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16665583#comment-16665583
]
ASF subversion and git services commented on IMPALA-7698:
-
Commit c1701074d6e94d98a43ab049ef807ac1b368180f in impala's branch
refs/heads/master from [~philip]
[ https://git-wip-us.apache.org/repos/asf?p=impala.git;h=c170107 ]
IMPALA-7698: Add centos support to bootstrap_system.
Largely, the changes involve conditionalizing some invocations to
account for differences between RH and Ubuntu. The trickiest bits were
timezone-related test errors (see below), postgresql permissions (need
to accept md5 passwords from localhost) and default ulimits (1024 user
processes/threads is not enough).
To test this, I built using test-with-docker. In additional to the
ulimit issue, I ran into the fact that /tmp needed 1777 permissions for
the postgresql socket, and entrypoint.sh had a few places that needed
special cases. At the moment, the data load ran fine, as did most of the
tests. I observed a test that relied on a python2.7-ism fail, which is
part of the point of this.
In the course of development, I encountered a handful of tests fail with
"Encounter parse error: failed to open /usr/share/zoneinfo/GMT-08:00 -
No such file or directory.", which was reproduced as follows:
[localhost:21000] default> use functional_orc_def; select * from alltypes;
...
WARNINGS: Encounter parse error: failed to open
/usr/share/zoneinfo/GMT-08:00 - No such file or directory.
With Quanlong's help, I learned what was happening. test-with-docker was
translating my time zone (America/Los_Angeles) to US/Pacific-New,
because realpath(/etc/localtime) = US/Pacific-New. This timezone exists
in centos:6, so that wasn't a problem. However, this timezone does not
exist in the package "tzdata-java", which is the copy of the timezone
information used by Java. (There are bugs here that may have been fixed
in centos:7.) As a result, when ORC asks (by using
TimeZone.getDefault().getID()) the JDK
(src/solaris/native/java/util/TimeZone_md.c) for the default timezone,
it can't find the same name as /etc/localtime points to in its
repository and defaults to "GMT-08:00". This string then gets written
into the ORC files generated by Hive as part of data load, and then the
C++ library can't read them. This is fixed by changing "realpath"
to "readlink" in test-with-docker.py.
centos:7 is not addressed by this change. The move to systemd makes
"service sshd start" (and the same for postgresql) not work, and
additional care needs to be done to work around that.
This change is a joint effort with Laszlo Gaal.
Change-Id: Id54294d7607f51de87a9de373dcfc4a33f4bedf5
Reviewed-on: http://gerrit.cloudera.org:8080/11731
Reviewed-by: Philip Zeyliger
Tested-by: Impala Public Jenkins
> Add centos/redhat 6/7 support to bootstrap_system.sh
>
>
> Key: IMPALA-7698
> URL: https://issues.apache.org/jira/browse/IMPALA-7698
> Project: IMPALA
> Issue Type: Task
> Components: Infrastructure
>Reporter: Philip Zeyliger
>Assignee: Philip Zeyliger
>Priority: Major
>
> {{bootstrap_system.sh}} currently only works on Ubuntu. Making it work on
> CentOS/Redhat would open the door to running automated tests on those
> platforms more readily, including using {{test-with-docker}}.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
-
To unsubscribe, e-mail: issues-all-unsubscr...@impala.apache.org
For additional commands, e-mail: issues-all-h...@impala.apache.org
