Hello Robert,
Thank you for quick response!
Indeed logs says the hadoop version is 2.4.1 this is probably because of
https://github.com/apache/flink/blob/b17a597dec80e590db2beedda446aa3cae9920dd/pom.xml#L96
How can I make 1.10 to work with my current hadoop version?
Regarding flink reporting in logs its 1.7.0
org.apache.flink.runtime.entrypoint.ClusterEntrypoint - Starting
YarnJobClusterEntrypoint (Version: 1.7.0
while I'm using 1.10 and this is application cluster (everything is bundled
and we don't have session cluster running).
Here are the whole dependencies list:
mvn dependency:tree | grep flink | cut -d'-' -f2-
org.apache.flink:flink-yarn_2.11:jar:1.10.0:runtime
org.apache.flink:flink-clients_2.11:jar:1.10.0:compile
org.apache.flink:flink-optimizer_2.11:jar:1.10.0:compile
org.apache.flink:flink-shaded-hadoop-2:jar:2.4.1-9.0:runtime
org.apache.flink:force-shading:jar:1.10.0:compile
org.apache.flink:flink-runtime_2.11:jar:1.10.0:runtime
org.apache.flink:flink-core:jar:1.10.0:compile
org.apache.flink:flink-annotations:jar:1.10.0:compile
org.apache.flink:flink-metrics-core:jar:1.10.0:compile
org.apache.flink:flink-java:jar:1.10.0:compile
org.apache.flink:flink-queryable-state-client-java:jar:1.10.0:runtime
org.apache.flink:flink-hadoop-fs:jar:1.10.0:runtime
org.apache.flink:flink-shaded-netty:jar:4.1.39.Final-9.0:compile
org.apache.flink:flink-shaded-guava:jar:18.0-9.0:compile
org.apache.flink:flink-shaded-asm-7:jar:7.1-9.0:compile
org.apache.flink:flink-shaded-jackson:jar:2.10.1-9.0:compile
org.apache.flink:flink-jdbc_2.11:jar:1.10.0:compile
org.apache.flink:flink-hbase_2.11:jar:1.10.0:compile
org.apache.flink:flink-runtime-web_2.11:jar:1.10.0:compile
As you can see all flink related libs are 1.10.
Can you please tell which class in flinks identifies the version(I'll try
to debug it locally)?
Regards,
Vitaliy
On Mon, Mar 30, 2020 at 5:10 PM Robert Metzger <rmetz...@apache.org> wrote:
> Hey Vitaliy,
> is it okay for you if we keep the discussion on the list, so that others
> can chime in to help, and that Google can index the conversation, in case
> somebody else has a similar problem?
>
> I just checked, and Flink on YARN in Flink 1.10 does set the
> property correctly. Maybe in Flink 1.7, accessing the logs in the web ui
> was not yet supported.
>
> You said in your email, that you are using Flink 1.10, however, your logs
> state that you are running Flink 1.7.0.
> It also seems that you have the Hadoop 2.4.1 dependencies of Flink, but
> your Hadoop environment is Hadoop 2.7.3. I believe this error is caused by
> that version mismatch:
>
> Caused by: java.lang.IllegalAccessError: tried to access method
>> org.apache.hadoop.yarn.client.ConfiguredRMFailoverProxyProvider.getProxyInternal()Ljava/lang/Object;
>> from class
>> org.apache.hadoop.yarn.client.RequestHedgingRMFailoverProxyProvider
>
>
>
>
> On Mon, Mar 30, 2020 at 1:58 PM Vitaliy Semochkin <vitaliy...@gmail.com>
> wrote:
>
>> Hello Robert,
>> >Where exactly are you seeing the "Log file environment variable
>> 'log.file' is not set." message?
>> I get this message when I check yarn logs.
>>
>> >Can you post some context around it? (is this shown from the command
>> line? what are the arguments? is it shown in a file?
>> I'm creating an application cluster from a java application using flink
>> 1.10. I didn't have this issue with 1.8.1 version,
>> however when I upgraded configuration slightly changed, e.g.
>> ClusterSpecification taskManagerMemoryMB is now ignored, and should be set
>> via fink Configuration.
>> (though it's value still validated, but no longer used after that).
>> My main issue, is that Application seems to fail to start properly, it
>> seems that JobMaster fails to connect to ResourceManager, but I can't
>> figure out why.
>> The yarn log is attached.
>>
>> I'll appreciate if you tell me to which direction I should dig.
>>
>> Regards,
>> Vitaliy
>>
>>
>> On Mon, Mar 30, 2020 at 12:00 PM Robert Metzger <rmetz...@apache.org>
>> wrote:
>>
>>> Hey,
>>> which Flink version are you using?
>>>
>>> Where exactly are you seeing the "Log file environment variable
>>> 'log.file' is not set." message? Can you post some context around it? (is
>>> this shown from the command line? what are the arguments? is it shown in a
>>> file?
>>>
>>> Usually, the "log.file" property is used to pass the name of the log
>>> file into the log4j configuration. If this property is not set, I have to
>>> assume that you are using modified or custom scripts, or you are executing
>>> Flink in an environment that fails to set the property.
>>>
>>> When running Flink on YARN, the JobManager logs are stored on the
>>> machine running the JobManager. The logs accessible through "yarn logs" are
>>> the same as you would see in the JM interface.
>>>
>>> Best,
>>> Robert
>>>
>>>
>>>
>>> On Sun, Mar 29, 2020 at 11:22 PM Vitaliy Semochkin <vitaliy...@gmail.com>
>>> wrote:
>>>
>>>> Hello Yun,
>>>>
>>>> I see this error reported by:
>>>> *org.apache.flink.runtime.webmonitor.WebMonitorUtils* - *JobManager
>>>> log files are unavailable in the web dashboard. Log file location not found
>>>> in environment variable 'log.file' or configuration key 'Key:
>>>> 'web.log.path' , default: null (fallback keys:
>>>> [{key=jobmanager.web.log.path, isDeprecated=true}])'.*
>>>>
>>>> I wonder where the JobManager files are stored in case running on a
>>>> YARN cluster?
>>>> Are these logs same to those I get via yarn logs -applicationId?
>>>>
>>>> Regards,
>>>> Vitaliy
>>>>
>>>>
>>>>
>>>> On Sun, Mar 29, 2020 at 8:24 PM Yun Tang <myas...@live.com> wrote:
>>>>
>>>>> Hi Vitaliy
>>>>>
>>>>> Property of 'log.file' would be configured if you have uploaded
>>>>> 'logback.xml' or 'log4j.properties' [1].
>>>>> The file would contain logs of job manager or task manager which is
>>>>> decided by the component itself. And as you can see, this is only a local
>>>>> file path, I am afraid this cannot understand hdfs paths.
>>>>>
>>>>>
>>>>> [1]
>>>>> https://github.com/apache/flink/blob/ae3b0ff80b93a83a358ab474060473863d2c30d6/flink-runtime/src/main/java/org/apache/flink/runtime/clusterframework/BootstrapTools.java#L420
>>>>>
>>>>> Best
>>>>> Yun Tang
>>>>> ------------------------------
>>>>> *From:* Vitaliy Semochkin <vitaliy...@gmail.com>
>>>>> *Sent:* Sunday, March 29, 2020 4:32
>>>>> *To:* user <user@flink.apache.org>
>>>>> *Subject:* Log file environment variable 'log.file' is not set.
>>>>>
>>>>> Hi,
>>>>>
>>>>> When I launch Flink Application Cluster I keep getting a message
>>>>> " Log file environment variable 'log.file' is not set."
>>>>>
>>>>> I use console logging via log4j
>>>>> and I read logs via yarn logs -applicationId ....
>>>>>
>>>>> What's the purpose of log.file property?
>>>>> What this file will contain and on which host should I search for the
>>>>> log?
>>>>> Does this property understands hdfs paths?
>>>>>
>>>>> Regards,
>>>>> Vitaliy
>>>>>
>>>>
