[jira] [Created] (FLINK-11344) Show All Execution Attempt Information on Flink Web Dashboard

[BoWang (JIRA)]

BoWang created FLINK-11344:
--

 Summary: Show All Execution Attempt Information on Flink Web 
Dashboard
 Key: FLINK-11344
 URL: https://issues.apache.org/jira/browse/FLINK-11344
 Project: Flink
  Issue Type: Improvement
  Components: Webfrontend
Reporter: BoWang
Assignee: BoWang


Currently, only one Execution Attempt of each sub-task is shown in web 
dashboard, thus, only the succeed Attempt is shown when failover occurs. This 
may be inconvenient to rapidly locate the failure reasons of failed Attempts



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

Re: 1.7.0 build failure, could not get required dependency from maven central.

[Chesnay Schepler]

The hadoop uber jar is not deployed to maven. You have to build it 
locally if you want to build flink-dist yourself.


On 14.01.2019 15:22, jiaxl wrote:

Below is maven build error:

[ERROR] Failed to execute goal on project flink-dist_2.11: Could not resolve
dependencies for project org.apache.flink:flink-dist_2.11:jar:1.7.0: Failure
to find org.apache.flink:flink-shaded-hadoop2-uber:jar:1.7.0 in
https://repo.maven.apache.org/maven2 was cached in the local repository,
resolution will not be reattempted until the update interval of central has
elapsed or updates are forced.

I didn't change anything, any clues?

Thanks.



--
Sent from: http://apache-flink-mailing-list-archive.1008284.n3.nabble.com/

Re: [NOTICE] Mandatory migration of git repositories to gitbox.apache.org

[Tzu-Li (Gordon) Tai]

Makes sense if we haven't been using them.
+1 to drop.

On Thu, Jan 3, 2019 at 2:45 PM Aljoscha Krettek  wrote:

> Sounds good.
>
> > On 3. Jan 2019, at 14:27, Chesnay Schepler  wrote:
> >
> > Since neither of these repositories are in use (flink-libraries is
> empty, and incubator-flink is 3+ years old) we could just drop them I
> suppose.
> >
> > Any objections?
> >
> > On 03.01.2019 14:18, Apache Infrastructure Team wrote:
> >> Hello, flink folks.
> >> As stated earlier in 2018, all git repositories must be migrated from
> >> the git-wip-us.apache.org URL to gitbox.apache.org, as the old service
> >> is being decommissioned. Your project is receiving this email because
> >> you still have repositories on git-wip-us that needs to be migrated.
> >>
> >> The following repositories on git-wip-us belong to your project:
> >> - incubator-flink.git
> >> - flink-libraries.git
> >>
> >>
> >> We are now entering the mandated (coordinated) move stage of the
> roadmap,
> >> and you are asked to please coordinate migration with the Apache
> >> Infrastructure Team before February 7th. All repositories not migrated
> >> on February 7th will be mass migrated without warning, and we'd
> appreciate
> >> it if we could work together to avoid a big mess that day :-).
> >>
> >> Moving to gitbox means you will get full write access on GitHub as well,
> >> and be able to close/merge pull requests and much more.
> >>
> >> To have your repositories moved, please follow these steps:
> >>
> >> - Ensure consensus on the move (a link to a lists.apache.org thread
> will
> >>  suffice for us as evidence).
> >> - Create a JIRA ticket at https://issues.apache.org/jira/browse/INFRA
> >>
> >> Your migration should only take a few minutes. If you wish to migrate
> >> at a specific time of day or date, please do let us know in the ticket.
> >>
> >> As always, we appreciate your understanding and patience as we move
> >> things around and work to provide better services and features for
> >> the Apache Family.
> >>
> >> Should you wish to contact us with feedback or questions, please do so
> >> at: us...@infra.apache.org.
> >>
> >>
> >> With regards,
> >> Apache Infrastructure
> >>
> >>
> >
>
>

Re: [NOTICE] Mandatory migration of git repositories to gitbox.apache.org

[Chesnay Schepler]

I've filed https://issues.apache.org/jira/browse/INFRA-17666.

On 16.01.2019 10:32, Tzu-Li (Gordon) Tai wrote:

Makes sense if we haven't been using them.
+1 to drop.

On Thu, Jan 3, 2019 at 2:45 PM Aljoscha Krettek > wrote:


Sounds good.

> On 3. Jan 2019, at 14:27, Chesnay Schepler mailto:ches...@apache.org>> wrote:
>
> Since neither of these repositories are in use (flink-libraries
is empty, and incubator-flink is 3+ years old) we could just drop
them I suppose.
>
> Any objections?
>
> On 03.01.2019 14:18, Apache Infrastructure Team wrote:
>> Hello, flink folks.
>> As stated earlier in 2018, all git repositories must be
migrated from
>> the git-wip-us.apache.org  URL to
gitbox.apache.org , as the old service
>> is being decommissioned. Your project is receiving this email
because
>> you still have repositories on git-wip-us that needs to be
migrated.
>>
>> The following repositories on git-wip-us belong to your project:
>> - incubator-flink.git
>> - flink-libraries.git
>>
>>
>> We are now entering the mandated (coordinated) move stage of
the roadmap,
>> and you are asked to please coordinate migration with the Apache
>> Infrastructure Team before February 7th. All repositories not
migrated
>> on February 7th will be mass migrated without warning, and we'd
appreciate
>> it if we could work together to avoid a big mess that day :-).
>>
>> Moving to gitbox means you will get full write access on GitHub
as well,
>> and be able to close/merge pull requests and much more.
>>
>> To have your repositories moved, please follow these steps:
>>
>> - Ensure consensus on the move (a link to a lists.apache.org
 thread will
>>  suffice for us as evidence).
>> - Create a JIRA ticket at
https://issues.apache.org/jira/browse/INFRA
>>
>> Your migration should only take a few minutes. If you wish to
migrate
>> at a specific time of day or date, please do let us know in the
ticket.
>>
>> As always, we appreciate your understanding and patience as we move
>> things around and work to provide better services and features for
>> the Apache Family.
>>
>> Should you wish to contact us with feedback or questions,
please do so
>> at: us...@infra.apache.org .
>>
>>
>> With regards,
>> Apache Infrastructure
>>
>>
>

[jira] [Created] (FLINK-11345) BucketingSinkFaultToleranceITCase failed on Travis

[Chesnay Schepler (JIRA)]

Chesnay Schepler created FLINK-11345:


 Summary: BucketingSinkFaultToleranceITCase failed on Travis
 Key: FLINK-11345
 URL: https://issues.apache.org/jira/browse/FLINK-11345
 Project: Flink
  Issue Type: Bug
  Components: Streaming, Tests
Affects Versions: 1.8.0
Reporter: Chesnay Schepler


https://travis-ci.org/apache/flink/jobs/480128907

{code}
00:16:03.610 [ERROR] Tests run: 1, Failures: 1, Errors: 0, Skipped: 0, Time 
elapsed: 18.564 s <<< FAILURE! - in 
org.apache.flink.streaming.connectors.fs.bucketing.BucketingSinkFaultToleranceITCase
00:16:03.611 [ERROR] 
runCheckpointedProgram(org.apache.flink.streaming.connectors.fs.bucketing.BucketingSinkFaultToleranceITCase)
  Time elapsed: 6.99 s  <<< FAILURE!
java.lang.AssertionError: Read line does not match expected pattern.
at 
org.apache.flink.streaming.connectors.fs.bucketing.BucketingSinkFaultToleranceITCase.postSubmit(BucketingSinkFaultToleranceITCase.java:182)
{code}



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

Re: [DISCUSS] Support Interactive Programming in Flink Table API

[Piotr Nowojski]

Hey Becket,

+1 From my side

Piotrek

> On 14 Jan 2019, at 14:43, Becket Qin  wrote:
> 
> Hi Seth,
> 
> Thanks for the feedback. Re-caching makes sense to me. Piotr and I had some
> offline discussion and we generally reached consensus on the following API:
> 
> {
>  /**
>* Cache this table to builtin table service or the specified customized
> table service.
>*
>* This method provides a hint to Flink that the current table maybe
> reused later so a
>* cache should be created to avoid regenerating this table.
>*
>* The following code snippet gives an example of how this method could
> be used.
>*
>* {{{
>*   val t = tEnv.fromCollection(data).as('country, 'color, 'count)
>*
>*   val t1 = t.filter('count < 100).cache()
>*   // t1 is cached after it is computed for the first time.
>*   val x = t1.collect().size
>*
>*   // When t1 is used again to compute t2, it may not be re-computed.
>*   val t2 = t1.groupBy('country).select('country, 'count.sum as 'sum)
>*   val res2 = t2.collect()
>*   res2.foreach(println)
>*
>*   // Similarly when t1 is used again to compute t3, it may not be
> re-computed.
>*   val t3 = t1.groupBy('color).select('color, 'count.avg as 'avg)
>*   val res3 = t3.collect()
>*   res3.foreach(println)
>*
>* }}}
>*
>* @note Flink optimizer may decide to not use the cache if doing that
> will accelerate the
>* processing, or if the cache is no longer available for reasons such
> as the cache has
>* been invalidated.
>* @note The table cache could be create lazily. That means the cache
> may be created at
>* the first time when the cached table is computed.
>* @note The table cache will be cleared when the user program exits.
>*
>* @return the current table with a cache hint. The original table
> reference is not modified
>*   by the execution of this method.
>*/
>  def cache(): Table
> 
>  /**
>* Manually invalidate the cache of this table to release the physical
> resources. Users are
>* not required to invoke this method to release physical resource
> unless they want to. The
>* table caches are cleared when user program exits.
>*
>* @note After invalidated, the cache may be re-created if this table is
> used again.
>*/
>  def invalidateCache(): Unit
> }
> 
> In the future, after we introduce automatic caching, the table may also be
> automatically cached.
> 
> In summary the final state we are looking at is following:
> 1. A table could be cached either manually or automatically.
> 2. If cache exists, Flink may or may not use it, depending on whether that
> will accelerate the execution.
> 3. In some rare use cases, an hint of could be used to explicitly ask Flink
> to ignore cache.
> 
> I'll document all the discussions we have had around the API. If there is
> no further concerns over this API, I'll convert it to a FLIP.
> 
> Thanks,
> 
> Jiangjie (Becket) Qin
> 
> On Thu, Jan 10, 2019 at 9:08 PM Seth Wiesman  wrote:
> 
>> I spoke to Piotr a little bit offline and I wanted to comment with a
>> summary of our discussion and what I believe is most intuitive cache model
>> from a users perspective.
>> 
>> (I am making up some class names here, not looking to bike shed feel free
>> to change the names how ever you see fit).
>> 
>> A cache is by definition an optimization, something used to store
>> intermediate results for faster / more performant downstream computation.
>> Therefore, as a Flink user I would not expect it to change the semantics of
>> my application, I would expect it to be rebuildable, and I do not expect to
>> know how it works under the hood. With there principles in mind I feel the
>> most intuitive api would be as follows:
>> 
>> // Some table
>> Table a = . . .
>> 
>> // Signal that we would like to cache the table
>> // this is lazy and does not force any computation.
>> CachedTable cachedA = a.cache()
>> 
>> // The first operation against the cache.
>> // This count will trigger reading input
>> // data and building the cache.
>> cachedA.count()
>> 
>> // Operates against the cache, no operations
>> // before a.cache are performed.
>> cachedA.sum()
>> 
>> // This does not operate against the cache,
>> // it will trigger reading data from source
>> // and performing a full computation
>> a.min()
>> 
>> // Invalidates the cache, releasing all
>> // underlying resources
>> cachedA.invalidateCache()
>> 
>> // Rebuilds the cache. Since caches are recomputable
>> // this should not be an error, it will simply be a more
>> // expensive operation than if we had not invalidated the cache.
>> cachedA.min()
>> 
>> This model leads to 2 nice properties:
>> 
>> 1) The same cache can be shared across multiple invocations of
>> Table#cache. Because the cache can always be rebuilt one code path
>> invalidating the cache will not break others. Cache’s are simply and
>> optimization and rebuilding the cache is

Re: [NOTICE] Mandatory migration of git repositories to gitbox.apache.org

[Ufuk Celebi]

Thanks Chesnay.

+1 to drop since we are not using them (flink-libraries is empty and
incubator-flink redirects to flink in GitHub).

– Ufuk

On Wed, Jan 16, 2019 at 10:38 AM Chesnay Schepler  wrote:
>
> I've filed https://issues.apache.org/jira/browse/INFRA-17666.
>
> On 16.01.2019 10:32, Tzu-Li (Gordon) Tai wrote:
> > Makes sense if we haven't been using them.
> > +1 to drop.
> >
> > On Thu, Jan 3, 2019 at 2:45 PM Aljoscha Krettek  > > wrote:
> >
> > Sounds good.
> >
> > > On 3. Jan 2019, at 14:27, Chesnay Schepler  > > wrote:
> > >
> > > Since neither of these repositories are in use (flink-libraries
> > is empty, and incubator-flink is 3+ years old) we could just drop
> > them I suppose.
> > >
> > > Any objections?
> > >
> > > On 03.01.2019 14:18, Apache Infrastructure Team wrote:
> > >> Hello, flink folks.
> > >> As stated earlier in 2018, all git repositories must be
> > migrated from
> > >> the git-wip-us.apache.org  URL to
> > gitbox.apache.org , as the old service
> > >> is being decommissioned. Your project is receiving this email
> > because
> > >> you still have repositories on git-wip-us that needs to be
> > migrated.
> > >>
> > >> The following repositories on git-wip-us belong to your project:
> > >> - incubator-flink.git
> > >> - flink-libraries.git
> > >>
> > >>
> > >> We are now entering the mandated (coordinated) move stage of
> > the roadmap,
> > >> and you are asked to please coordinate migration with the Apache
> > >> Infrastructure Team before February 7th. All repositories not
> > migrated
> > >> on February 7th will be mass migrated without warning, and we'd
> > appreciate
> > >> it if we could work together to avoid a big mess that day :-).
> > >>
> > >> Moving to gitbox means you will get full write access on GitHub
> > as well,
> > >> and be able to close/merge pull requests and much more.
> > >>
> > >> To have your repositories moved, please follow these steps:
> > >>
> > >> - Ensure consensus on the move (a link to a lists.apache.org
> >  thread will
> > >>  suffice for us as evidence).
> > >> - Create a JIRA ticket at
> > https://issues.apache.org/jira/browse/INFRA
> > >>
> > >> Your migration should only take a few minutes. If you wish to
> > migrate
> > >> at a specific time of day or date, please do let us know in the
> > ticket.
> > >>
> > >> As always, we appreciate your understanding and patience as we move
> > >> things around and work to provide better services and features for
> > >> the Apache Family.
> > >>
> > >> Should you wish to contact us with feedback or questions,
> > please do so
> > >> at: us...@infra.apache.org .
> > >>
> > >>
> > >> With regards,
> > >> Apache Infrastructure
> > >>
> > >>
> > >
> >
>

Data not getting passed between operators

[Ramya Ramamurthy]

Hi

I have a Flink 1.7 with Kafka 0.11 and ES 6.5 setup.

I can see the Flink Kafka Consumer consuming messages, but these are not
passed on to the next level, that is the elasticsearch sink. Unable to find
any logs relevant to this.

Logs about my kafka consumers

2019-01-16 17:28:05,860 DEBUG
org.apache.flink.kafka011.shaded.org.apache.kafka.clients.consumer.KafkaConsumer
- Committing offsets:
{legitimatestream-16=OffsetAndMetadata{offset=203252637, metadata=''},
legitimatestream-18=OffsetAndMetadata{offset=203236011, metadata=''},
legitimatestream-20=OffsetAndMetadata{offset=203237190, metadata=''},
legitimatestream-22=OffsetAndMetadata{offset=203273504, metadata=''},
legitimatestream-8=OffsetAndMetadata{offset=203251672, metadata=''},
legitimatestream-10=OffsetAndMetadata{offset=203235871, metadata=''},
legitimatestream-12=OffsetAndMetadata{offset=203242970, metadata=''},
legitimatestream-14=OffsetAndMetadata{offset=203269129, metadata=''},
legitimatestream-0=OffsetAndMetadata{offset=203247420, metadata=''},
legitimatestream--9=OffsetAndMetadata{offset=203226435, metadata=''},
legitimatestream-11=OffsetAndMetadata{offset=203259207, metadata=''},
legitimatestream-13=OffsetAndMetadata{offset=203262566, metadata=''},
legitimatestream-1=OffsetAndMetadata{offset=203230950, metadata=''},
legitimatestream-3=OffsetAndMetadata{offset=203260289, metadata=''},
legitimatestream-5=OffsetAndMetadata{offset=203285827, metadata=''},
legitimatestream-24=OffsetAndMetadata{offset=203240761, metadata=''},
legitimatestream-26=OffsetAndMetadata{offset=203254649, metadata=''},
legitimatestream-28=OffsetAndMetadata{offset=203265863, metadata=''}}

I am unable to see any logs regarding passing this data to the next level,
which is the Table Query, followed by sink.

Can anyone help me out with why this might happen, or am i missing
something ??

Here is the snippet of my code:
StreamTableEnvironment tableEnv = TableEnvironment.getTableEnvironment(env);

tableEnv.connect(new Kafka()
.version("0.11")
.topic(params.getRequired("read-topic"))
.property("auto.offset.reset", "latest")
.property("group.id", params.getRequired("group.id"))
.property("bootstrap.servers",
params.getRequired("bootstrap.servers")))
.withSchema(new Schema()
.field("sid", Types.STRING())
.field("_zpsbd6", Types.STRING())
.field("r1", Types.STRING())
.field("r2", Types.STRING())
.field("r5", Types.STRING())
.field("r10", Types.STRING())
.field("isBot", Types.BOOLEAN())
.field("botcode", Types.STRING())
.field("ts", Types.SQL_TIMESTAMP())
/*.field("sensor", Types.STRING())
.field("temp", Types.LONG())
.field("ts", Types.SQL_TIMESTAMP())*/
.rowtime(new Rowtime()
.timestampsFromField("_zpsbda")
.watermarksPeriodicBounded(5000)
)
)
.withFormat(new Json().deriveSchema())
.inAppendMode()
.registerTableSource("sourceTopic");
Table query = tableEnv.sqlQuery("SELECT sid, _zpsbd6 as ip, COUNT(*) as
total_hits, " +
"TUMBLE_START(ts, INTERVAL '1' MINUTE) as
tumbleStart, " +
"TUMBLE_END(ts, INTERVAL '1' MINUTE) as
tumbleEnd FROM sourceTopic " +
"WHERE r1='true' or r2='true' or r5='true'
or r10='true' and isBot='true'" +
"GROUP BY TUMBLE(ts, INTERVAL '1' MINUTE),
sid,  _zpsbd6");

TypeInformation schema = query.getSchema().toRowType();
SerializationSchema serializationSchema = new
JsonRowSerializationSchema(schema);
DataStream ds = tableEnv.toAppendStream(query,
Row.class);
ds.print();

List httpHosts = new ArrayList<>();
httpHosts.add(new HttpHost("127.0.0.1", 9200, "http"));
ElasticsearchSink.Builder esSinkBuilder = new
ElasticsearchSink.Builder<>(
httpHosts,
new ElasticsearchSinkFunction() {

public IndexRequest
createIndexRequest(Row row) {
byte[] document =
serializationSchema.serialize(row);

return new
IndexRequest("prod", "logs")

.source(document, XContentType.JSON);

}

   

Re: [NOTICE] Mandatory migration of git repositories to gitbox.apache.org

[Stephan Ewen]

+1 to drop those repos

On Wed, Jan 16, 2019 at 12:08 PM Ufuk Celebi  wrote:

> Thanks Chesnay.
>
> +1 to drop since we are not using them (flink-libraries is empty and
> incubator-flink redirects to flink in GitHub).
>
> – Ufuk
>
> On Wed, Jan 16, 2019 at 10:38 AM Chesnay Schepler 
> wrote:
> >
> > I've filed https://issues.apache.org/jira/browse/INFRA-17666.
> >
> > On 16.01.2019 10:32, Tzu-Li (Gordon) Tai wrote:
> > > Makes sense if we haven't been using them.
> > > +1 to drop.
> > >
> > > On Thu, Jan 3, 2019 at 2:45 PM Aljoscha Krettek  > > > wrote:
> > >
> > > Sounds good.
> > >
> > > > On 3. Jan 2019, at 14:27, Chesnay Schepler  > > > wrote:
> > > >
> > > > Since neither of these repositories are in use (flink-libraries
> > > is empty, and incubator-flink is 3+ years old) we could just drop
> > > them I suppose.
> > > >
> > > > Any objections?
> > > >
> > > > On 03.01.2019 14:18, Apache Infrastructure Team wrote:
> > > >> Hello, flink folks.
> > > >> As stated earlier in 2018, all git repositories must be
> > > migrated from
> > > >> the git-wip-us.apache.org  URL to
> > > gitbox.apache.org , as the old service
> > > >> is being decommissioned. Your project is receiving this email
> > > because
> > > >> you still have repositories on git-wip-us that needs to be
> > > migrated.
> > > >>
> > > >> The following repositories on git-wip-us belong to your project:
> > > >> - incubator-flink.git
> > > >> - flink-libraries.git
> > > >>
> > > >>
> > > >> We are now entering the mandated (coordinated) move stage of
> > > the roadmap,
> > > >> and you are asked to please coordinate migration with the Apache
> > > >> Infrastructure Team before February 7th. All repositories not
> > > migrated
> > > >> on February 7th will be mass migrated without warning, and we'd
> > > appreciate
> > > >> it if we could work together to avoid a big mess that day :-).
> > > >>
> > > >> Moving to gitbox means you will get full write access on GitHub
> > > as well,
> > > >> and be able to close/merge pull requests and much more.
> > > >>
> > > >> To have your repositories moved, please follow these steps:
> > > >>
> > > >> - Ensure consensus on the move (a link to a lists.apache.org
> > >  thread will
> > > >>  suffice for us as evidence).
> > > >> - Create a JIRA ticket at
> > > https://issues.apache.org/jira/browse/INFRA
> > > >>
> > > >> Your migration should only take a few minutes. If you wish to
> > > migrate
> > > >> at a specific time of day or date, please do let us know in the
> > > ticket.
> > > >>
> > > >> As always, we appreciate your understanding and patience as we
> move
> > > >> things around and work to provide better services and features
> for
> > > >> the Apache Family.
> > > >>
> > > >> Should you wish to contact us with feedback or questions,
> > > please do so
> > > >> at: us...@infra.apache.org .
> > > >>
> > > >>
> > > >> With regards,
> > > >> Apache Infrastructure
> > > >>
> > > >>
> > > >
> > >
> >
>

Re: [NOTICE] Mandatory migration of git repositories to gitbox.apache.org

[Chesnay Schepler]

The repositories have been dropped.

On 16.01.2019 14:12, Stephan Ewen wrote:

+1 to drop those repos

On Wed, Jan 16, 2019 at 12:08 PM Ufuk Celebi > wrote:


Thanks Chesnay.

+1 to drop since we are not using them (flink-libraries is empty and
incubator-flink redirects to flink in GitHub).

– Ufuk

On Wed, Jan 16, 2019 at 10:38 AM Chesnay Schepler
mailto:ches...@apache.org>> wrote:
>
> I've filed https://issues.apache.org/jira/browse/INFRA-17666.
>
> On 16.01.2019 10:32, Tzu-Li (Gordon) Tai wrote:
> > Makes sense if we haven't been using them.
> > +1 to drop.
> >
> > On Thu, Jan 3, 2019 at 2:45 PM Aljoscha Krettek
mailto:aljos...@apache.org>
> > >> wrote:
> >
> > Sounds good.
> >
> > > On 3. Jan 2019, at 14:27, Chesnay Schepler
mailto:ches...@apache.org>
> > >>
wrote:
> > >
> > > Since neither of these repositories are in use
(flink-libraries
> > is empty, and incubator-flink is 3+ years old) we could
just drop
> > them I suppose.
> > >
> > > Any objections?
> > >
> > > On 03.01.2019 14:18, Apache Infrastructure Team wrote:
> > >> Hello, flink folks.
> > >> As stated earlier in 2018, all git repositories must be
> > migrated from
> > >> the git-wip-us.apache.org
  URL to
> > gitbox.apache.org 
, as the old service
> > >> is being decommissioned. Your project is receiving this
email
> > because
> > >> you still have repositories on git-wip-us that needs to be
> > migrated.
> > >>
> > >> The following repositories on git-wip-us belong to your
project:
> > >> - incubator-flink.git
> > >> - flink-libraries.git
> > >>
> > >>
> > >> We are now entering the mandated (coordinated) move
stage of
> > the roadmap,
> > >> and you are asked to please coordinate migration with
the Apache
> > >> Infrastructure Team before February 7th. All
repositories not
> > migrated
> > >> on February 7th will be mass migrated without warning,
and we'd
> > appreciate
> > >> it if we could work together to avoid a big mess that
day :-).
> > >>
> > >> Moving to gitbox means you will get full write access
on GitHub
> > as well,
> > >> and be able to close/merge pull requests and much more.
> > >>
> > >> To have your repositories moved, please follow these steps:
> > >>
> > >> - Ensure consensus on the move (a link to a
lists.apache.org 
> >  thread will
> > >>  suffice for us as evidence).
> > >> - Create a JIRA ticket at
> > https://issues.apache.org/jira/browse/INFRA
> > >>
> > >> Your migration should only take a few minutes. If you
wish to
> > migrate
> > >> at a specific time of day or date, please do let us
know in the
> > ticket.
> > >>
> > >> As always, we appreciate your understanding and
patience as we move
> > >> things around and work to provide better services and
features for
> > >> the Apache Family.
> > >>
> > >> Should you wish to contact us with feedback or questions,
> > please do so
> > >> at: us...@infra.apache.org
 >.
> > >>
> > >>
> > >> With regards,
> > >> Apache Infrastructure
> > >>
> > >>
> > >
> >
>

[jira] [Created] (FLINK-11346) Could not enable basic authentication with elastic search connector

[kannan (JIRA)]

kannan created FLINK-11346:
--

 Summary: Could not enable basic authentication with elastic search 
connector
 Key: FLINK-11346
 URL: https://issues.apache.org/jira/browse/FLINK-11346
 Project: Flink
  Issue Type: Bug
  Components: ElasticSearch Connector
Reporter: kannan


Hello Team, 

I am trying to add elasticsearch as sink and my current elasticsearch has 
enabled SSL and Basic authentication. 

I could not connect to elasticsearch with basic authentication. 

===

code:

val credentialsProvider = new BasicCredentialsProvider
 credentialsProvider.setCredentials(AuthScope.ANY, new 
UsernamePasswordCredentials("user", "password"))

esSinkBuilder.setRestClientFactory(
 restClientBuilder -> {
 restClientBuilder.setHttpClientConfigCallback(new 
RestClientBuilder.HttpClientConfigCallback()

{ override def customizeHttpClient(httpClientBuilder: HttpAsyncClientBuilder) = 
httpClientBuilder .setDefaultCredentialsProvider(credentialsProvider) }

)
 }
 )

===

 

Error Log:

Error:(70, 7) not found: value restClientBuilder
 restClientBuilder -> {

 

Application version used 

scala - 2.11

flink - 1.7

elasticsearch - 6.5

elasticsearch connector version - 2.11



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Created] (FLINK-11347) Optimize the ParquetAvroWriters factory

[Fokko Driesprong (JIRA)]

Fokko Driesprong created FLINK-11347:


 Summary: Optimize the ParquetAvroWriters factory
 Key: FLINK-11347
 URL: https://issues.apache.org/jira/browse/FLINK-11347
 Project: Flink
  Issue Type: Improvement
  Components: Formats
Affects Versions: 1.7.1
Reporter: Fokko Driesprong
Assignee: Fokko Driesprong
 Fix For: 1.8.0


In the ParquetAvroWriters the schema is first serialized to a string, and then 
back to a Schema, which is quite expensive to do. Therefore it makes sense to 
pass the schema to the writer directly.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Created] (FLINK-11348) Port YARNSessionCapacitySchedulerITCase to new code base

[Till Rohrmann (JIRA)]

Till Rohrmann created FLINK-11348:
-

 Summary: Port YARNSessionCapacitySchedulerITCase to new code base
 Key: FLINK-11348
 URL: https://issues.apache.org/jira/browse/FLINK-11348
 Project: Flink
  Issue Type: Sub-task
Reporter: vinoyang
Assignee: Till Rohrmann


{{YARNHighAvailabilityITCase}}, 

{{YARNSessionCapacitySchedulerITCase#testClientStartup,}} 

{{YARNSessionCapacitySchedulerITCase#testTaskManagerFailure}}



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Created] (FLINK-11349) Port CoordinatorShutdownTest to new code base

[Till Rohrmann (JIRA)]

Till Rohrmann created FLINK-11349:
-

 Summary: Port CoordinatorShutdownTest to new code base
 Key: FLINK-11349
 URL: https://issues.apache.org/jira/browse/FLINK-11349
 Project: Flink
  Issue Type: Sub-task
Reporter: Till Rohrmann


Port {{CoordinatorShutdownTest#testCoordinatorShutsDownOnFailure}} and 
{{CoordinatorShutdownTest#testCoordinatorShutsDownOnSuccess}} to new code base.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Created] (FLINK-11350) Remove JobClientActorRecoveryITCase

[Till Rohrmann (JIRA)]

Till Rohrmann created FLINK-11350:
-

 Summary: Remove JobClientActorRecoveryITCase
 Key: FLINK-11350
 URL: https://issues.apache.org/jira/browse/FLINK-11350
 Project: Flink
  Issue Type: Sub-task
Reporter: Till Rohrmann
Assignee: Till Rohrmann


Remove {{JobClientActorRecoveryITCase}} since it only tests legacy code.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Created] (FLINK-11351) Port JobManagerCleanupITCase to new code base

[Till Rohrmann (JIRA)]

Till Rohrmann created FLINK-11351:
-

 Summary: Port JobManagerCleanupITCase to new code base
 Key: FLINK-11351
 URL: https://issues.apache.org/jira/browse/FLINK-11351
 Project: Flink
  Issue Type: Sub-task
  Components: Tests
Reporter: Till Rohrmann


Port {{JobManagerCleanupITCase}} to new code base.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Created] (FLINK-11352) Check and port JobManagerHACheckpointRecoveryITCase to new code base if necessary

[Till Rohrmann (JIRA)]

Till Rohrmann created FLINK-11352:
-

 Summary: Check and port JobManagerHACheckpointRecoveryITCase to 
new code base if necessary
 Key: FLINK-11352
 URL: https://issues.apache.org/jira/browse/FLINK-11352
 Project: Flink
  Issue Type: Sub-task
  Components: Tests
Reporter: Till Rohrmann


Check and port {{JobManagerHACheckpointRecoveryITCase}} to new code base if 
necessary



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Created] (FLINK-11353) Check & port JobManagerHAJobGraphRecoveryITCase if necessary to new code base

[Till Rohrmann (JIRA)]

Till Rohrmann created FLINK-11353:
-

 Summary: Check & port JobManagerHAJobGraphRecoveryITCase if 
necessary to new code base
 Key: FLINK-11353
 URL: https://issues.apache.org/jira/browse/FLINK-11353
 Project: Flink
  Issue Type: Sub-task
  Components: Tests
Reporter: Till Rohrmann


Check & port {{JobManagerHAJobGraphRecoveryITCase}} if necessary to new code 
base



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Created] (FLINK-11354) Check and port JobManagerHARecoveryTest if necessary to new code base

[Till Rohrmann (JIRA)]

Till Rohrmann created FLINK-11354:
-

 Summary: Check and port JobManagerHARecoveryTest if necessary to 
new code base
 Key: FLINK-11354
 URL: https://issues.apache.org/jira/browse/FLINK-11354
 Project: Flink
  Issue Type: Sub-task
  Components: Tests
Reporter: Till Rohrmann


Check and port {{JobManagerHARecoveryTest}} if necessary to new code base.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Created] (FLINK-11355) Check and port JobManagerProcessReapingTest if necessary to new code base

[Till Rohrmann (JIRA)]

Till Rohrmann created FLINK-11355:
-

 Summary: Check and port JobManagerProcessReapingTest if necessary 
to new code base
 Key: FLINK-11355
 URL: https://issues.apache.org/jira/browse/FLINK-11355
 Project: Flink
  Issue Type: Sub-task
  Components: Tests
Reporter: Till Rohrmann


Check and port {{JobManagerProcessReapingTest}} if necessary to new code base.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Created] (FLINK-11356) Check and port JobManagerStartupTest to new code base if necessary

[Till Rohrmann (JIRA)]

Till Rohrmann created FLINK-11356:
-

 Summary: Check and port JobManagerStartupTest to new code base if 
necessary
 Key: FLINK-11356
 URL: https://issues.apache.org/jira/browse/FLINK-11356
 Project: Flink
  Issue Type: Sub-task
Reporter: Till Rohrmann


Check and port {{JobManagerStartupTest}} to new code base if necessary.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Created] (FLINK-11357) Check and port LeaderChangeJobRecoveryTest to new code base if necessary

[Till Rohrmann (JIRA)]

Till Rohrmann created FLINK-11357:
-

 Summary: Check and port LeaderChangeJobRecoveryTest to new code 
base if necessary
 Key: FLINK-11357
 URL: https://issues.apache.org/jira/browse/FLINK-11357
 Project: Flink
  Issue Type: Sub-task
  Components: Tests
Reporter: Till Rohrmann


Check and port {{LeaderChangeJobRecoveryTest}} to new code base if necessary.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Created] (FLINK-11358) Check and port LeaderChangeStateCleanupTest to new code base if necessary

[Till Rohrmann (JIRA)]

Till Rohrmann created FLINK-11358:
-

 Summary: Check and port LeaderChangeStateCleanupTest to new code 
base if necessary
 Key: FLINK-11358
 URL: https://issues.apache.org/jira/browse/FLINK-11358
 Project: Flink
  Issue Type: Sub-task
  Components: Tests
Reporter: Till Rohrmann


Check and port {{LeaderChangeStateCleanupTest}} to new code base if necessary.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Created] (FLINK-11359) Check and port LegacyAvroExternalJarProgramITCase to new code base if necessary

[Till Rohrmann (JIRA)]

Till Rohrmann created FLINK-11359:
-

 Summary: Check and port LegacyAvroExternalJarProgramITCase to new 
code base if necessary
 Key: FLINK-11359
 URL: https://issues.apache.org/jira/browse/FLINK-11359
 Project: Flink
  Issue Type: Sub-task
  Components: Tests
Reporter: Till Rohrmann


Check and port {{LegacyAvroExternalJarProgramITCase}} to new code base if 
necessary.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Created] (FLINK-11361) Check and port RecoveryITCase to new code base if necessary

[Till Rohrmann (JIRA)]

Till Rohrmann created FLINK-11361:
-

 Summary: Check and port RecoveryITCase to new code base if 
necessary
 Key: FLINK-11361
 URL: https://issues.apache.org/jira/browse/FLINK-11361
 Project: Flink
  Issue Type: Sub-task
  Components: Tests
Reporter: Till Rohrmann


Check and port {{RecoveryITCase}} to new code base if necessary.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Created] (FLINK-11360) Check and remove LocalFlinkMiniClusterITCase

[Till Rohrmann (JIRA)]

Till Rohrmann created FLINK-11360:
-

 Summary: Check and remove LocalFlinkMiniClusterITCase
 Key: FLINK-11360
 URL: https://issues.apache.org/jira/browse/FLINK-11360
 Project: Flink
  Issue Type: Sub-task
  Components: Tests
Reporter: Till Rohrmann


Check tests in {{LocalFlinkMiniClusterITCase}} whether they also apply to the 
{{MiniCluster}} and port if necessary. Afterwards remove 
{{LocalFlinkMiniClusterITCase}}.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Created] (FLINK-11363) Check and remove TaskManagerConfigurationTest

[Till Rohrmann (JIRA)]

Till Rohrmann created FLINK-11363:
-

 Summary: Check and remove TaskManagerConfigurationTest
 Key: FLINK-11363
 URL: https://issues.apache.org/jira/browse/FLINK-11363
 Project: Flink
  Issue Type: Sub-task
  Components: Tests
Reporter: Till Rohrmann


Check whether {{TaskManagerConfigurationTest}} contains any relevant tests for 
the new code base and then remove this test.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Created] (FLINK-11362) Check and port TaskManagerComponentsStartupShutdownTest to new code base if necessary

[Till Rohrmann (JIRA)]

Till Rohrmann created FLINK-11362:
-

 Summary: Check and port TaskManagerComponentsStartupShutdownTest 
to new code base if necessary
 Key: FLINK-11362
 URL: https://issues.apache.org/jira/browse/FLINK-11362
 Project: Flink
  Issue Type: Sub-task
  Components: Tests
Reporter: Till Rohrmann


Check and port {{TaskManagerComponentsStartupShutdownTest}} to new code base if 
necessary.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Created] (FLINK-11365) Check and port TaskManagerFailureRecoveryITCase to new code base if necessary

[Till Rohrmann (JIRA)]

Till Rohrmann created FLINK-11365:
-

 Summary: Check and port TaskManagerFailureRecoveryITCase to new 
code base if necessary
 Key: FLINK-11365
 URL: https://issues.apache.org/jira/browse/FLINK-11365
 Project: Flink
  Issue Type: Sub-task
  Components: Tests
Reporter: Till Rohrmann


Check and port {{TaskManagerFailureRecoveryITCase}} to new code base if 
necessary.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Created] (FLINK-11364) Check and port TaskManagerFailsITCase to new code base if necessary

[Till Rohrmann (JIRA)]

Till Rohrmann created FLINK-11364:
-

 Summary: Check and port TaskManagerFailsITCase to new code base if 
necessary
 Key: FLINK-11364
 URL: https://issues.apache.org/jira/browse/FLINK-11364
 Project: Flink
  Issue Type: Sub-task
  Components: Tests
Reporter: Till Rohrmann


Check and port {{TaskManagerFailsITCase}} to new code base if necessary.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Created] (FLINK-11366) Check and port TaskManagerMetricsTest to new code base if necessary

[Till Rohrmann (JIRA)]

Till Rohrmann created FLINK-11366:
-

 Summary: Check and port TaskManagerMetricsTest to new code base if 
necessary
 Key: FLINK-11366
 URL: https://issues.apache.org/jira/browse/FLINK-11366
 Project: Flink
  Issue Type: Sub-task
  Components: Tests
Reporter: Till Rohrmann


Check and port {{TaskManagerMetricsTest}} to new code base if necessary.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Created] (FLINK-11367) Check and port TaskManagerProcessReapingTestBase to new code base if necessary

[Till Rohrmann (JIRA)]

Till Rohrmann created FLINK-11367:
-

 Summary: Check and port TaskManagerProcessReapingTestBase to new 
code base if necessary
 Key: FLINK-11367
 URL: https://issues.apache.org/jira/browse/FLINK-11367
 Project: Flink
  Issue Type: Sub-task
  Components: Tests
Reporter: Till Rohrmann


Check and port {{TaskManagerProcessReapingTestBase}} to new code base if 
necessary.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Created] (FLINK-11368) Check and port TaskManagerStartupTest to new code base if necessary

[Till Rohrmann (JIRA)]

Till Rohrmann created FLINK-11368:
-

 Summary: Check and port TaskManagerStartupTest to new code base if 
necessary
 Key: FLINK-11368
 URL: https://issues.apache.org/jira/browse/FLINK-11368
 Project: Flink
  Issue Type: Sub-task
  Components: Tests
Reporter: Till Rohrmann


Check and port {{TaskManagerStartupTest}} to new code base if necessary.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Created] (FLINK-11369) Check and port ZooKeeperHAJobManagerTest to new code base if necessary

[Till Rohrmann (JIRA)]

Till Rohrmann created FLINK-11369:
-

 Summary: Check and port ZooKeeperHAJobManagerTest to new code base 
if necessary
 Key: FLINK-11369
 URL: https://issues.apache.org/jira/browse/FLINK-11369
 Project: Flink
  Issue Type: Sub-task
  Components: Tests
Reporter: Till Rohrmann


Check and port {{ZooKeeperHAJobManagerTest}} to new code base if necessary.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Created] (FLINK-11370) Check and port ZooKeeperLeaderElectionITCase to new code base if necessary

[Till Rohrmann (JIRA)]

Till Rohrmann created FLINK-11370:
-

 Summary: Check and port ZooKeeperLeaderElectionITCase to new code 
base if necessary
 Key: FLINK-11370
 URL: https://issues.apache.org/jira/browse/FLINK-11370
 Project: Flink
  Issue Type: Sub-task
  Components: Tests
Reporter: Till Rohrmann


Check and port {{ZooKeeperLeaderElectionITCase}} to new code base if necessary.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Created] (FLINK-11371) Close the AvroParquetReader after use

[Fokko Driesprong (JIRA)]

Fokko Driesprong created FLINK-11371:


 Summary: Close the AvroParquetReader after use
 Key: FLINK-11371
 URL: https://issues.apache.org/jira/browse/FLINK-11371
 Project: Flink
  Issue Type: Improvement
  Components: Formats
Affects Versions: 1.7.1
Reporter: Fokko Driesprong
Assignee: Fokko Driesprong
 Fix For: 1.8.0


The AvroParquetReader is not being closed



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Created] (FLINK-11372) Incorrect delegation of compatibility checks to new snapshots in CollectionSerializerConfigSnapshot

[Tzu-Li (Gordon) Tai (JIRA)]

Tzu-Li (Gordon) Tai created FLINK-11372:
---

 Summary: Incorrect delegation of compatibility checks to new 
snapshots in CollectionSerializerConfigSnapshot
 Key: FLINK-11372
 URL: https://issues.apache.org/jira/browse/FLINK-11372
 Project: Flink
  Issue Type: Bug
  Components: Type Serialization System
Reporter: Tzu-Li (Gordon) Tai
Assignee: Tzu-Li (Gordon) Tai
 Fix For: 1.8.0


In {{CollectionSerializerConfigSnapshot}}:
{code}
@Override
public TypeSerializerSchemaCompatibility 
resolveSchemaCompatibility(TypeSerializer newSerializer) {
if (newSerializer instanceof ListSerializer) {
ListSerializer newListSerializer = 
(ListSerializer) newSerializer;
ListSerializerSnapshot listSerializerSnapshot = new 
ListSerializerSnapshot<>(newListSerializer);

@SuppressWarnings("unchecked")
TypeSerializerSchemaCompatibility result = 
(TypeSerializerSchemaCompatibility)

listSerializerSnapshot.resolveSchemaCompatibility(newListSerializer);
return result;
} else {
return super.resolveSchemaCompatibility(newSerializer);
}
}
{code}

Compatibility check of {{ListSerializer}} is delegated to the new list 
serializer snapshot class, {{ListSerializerSnapshot}}.
However, it is incorrect to let the delegate wrap the new serializer (and 
therefore the new nested element serializer). By doing that, we're essentially 
checking compatibility of the new serializer with itself, whereas it should be 
checking compatibility with the restored serializer.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Created] (FLINK-11373) CliFrontend cuts off reason for error messages

[Maximilian Michels (JIRA)]

Maximilian Michels created FLINK-11373:
--

 Summary: CliFrontend cuts off reason for error messages
 Key: FLINK-11373
 URL: https://issues.apache.org/jira/browse/FLINK-11373
 Project: Flink
  Issue Type: Bug
  Components: Startup Shell Scripts
Affects Versions: 1.7.1, 1.6.3, 1.5.6
Reporter: Maximilian Michels


The CliFrontend seems to only print the first message in the strace trace and 
not any of its causes.

{noformat}
bin/flink run /non-existing/path
Could not build the program from JAR file.

Use the help option (-h or --help) to get help on the command.
{noformat}

Notice, the underlying cause of this message is FileNotFoundException.

Consider changing 
a) the error message for this particular case 
b) the way the stack trace messages are trimmed



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

Re: 1.7.0 build failure, could not get required dependency from maven central.

[Stephen]

You can try with the newest maven. I met a similar problem but solved by
updating to a new version maven



--
Sent from: http://apache-flink-mailing-list-archive.1008284.n3.nabble.com/

[jira] [Created] (FLINK-11374) Faiover add time range filter

[lining (JIRA)]

lining created FLINK-11374:
--

 Summary: Faiover add time range filter
 Key: FLINK-11374
 URL: https://issues.apache.org/jira/browse/FLINK-11374
 Project: Flink
  Issue Type: Improvement
  Components: REST, Webfrontend
Reporter: lining
Assignee: lining


Now failover just show limit size task failover latest time. If task has failed 
many time, we can not see the earlier time failover. Can we add filter by time 
to see failover which contains task attemp fail msg.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Created] (FLINK-11375) Concurrent modification to slot pool due to SlotSharingManager releaseSlot directly

[shuai.xu (JIRA)]

shuai.xu created FLINK-11375:


 Summary: Concurrent modification to slot pool due to 
SlotSharingManager releaseSlot directly 
 Key: FLINK-11375
 URL: https://issues.apache.org/jira/browse/FLINK-11375
 Project: Flink
  Issue Type: Bug
  Components: JobManager
Affects Versions: 1.7.1
Reporter: shuai.xu


In SlotPool, the AvailableSlots is lock free, so all access to it should in the 
main thread of SlotPool, and so all the public methods are called throw 
SlotPoolGateway except the releaseSlot directly called by SlotSharingManager. 
This may cause a ConcurrentModificationException.

 



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Created] (FLINK-11376) flink cli -yn -ys is not effect if (yn * ys)

[shengjk1 (JIRA)]

shengjk1 created FLINK-11376:


 Summary: flink cli  -yn -ys is not  effect if  (yn * 
ys)https://issues.apache.org/jira/browse/FLINK-11376
 Project: Flink
  Issue Type: Bug
Affects Versions: 1.7.1
 Environment: java: jdk1.8.0_151

flink: flink-1.7.1

CDH:CDH-5.13.1-1.cdh5.13.1.p0.2
Reporter: shengjk1
 Attachments: Main222.java, image-2019-01-17-14-25-34-206.png

Such as the title

if  (yn * ys)parallelism 
,the  yn、ys will effect. if code write   env.setParallelism(parallelism) ，the 
final application resources are yn multiples and ys  multiples. such  as 
parallelism=10,yn=1 ys=5,the final application resources:cpu cores=11 
containers=3

 

Reproduce for the convenience of bugs，offer codes



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Created] (FLINK-11377) AbstractYarnClusterDescriptor's validClusterSpecification is not final application resources from yarn if cli -yn -ys not effect

[shengjk1 (JIRA)]

shengjk1 created FLINK-11377:


 Summary: AbstractYarnClusterDescriptor's validClusterSpecification 
is not final application resources from yarn  if   cli  -yn -ys not effect
 Key: FLINK-11377
 URL: https://issues.apache.org/jira/browse/FLINK-11377
 Project: Flink
  Issue Type: Bug
Affects Versions: 1.7.1
 Environment: java: jdk1.8.0_151

flink: flink-1.7.1

CDH:CDH-5.13.1-1.cdh5.13.1.p0.2
Reporter: shengjk1
 Attachments: image-2019-01-17-14-57-24-060.png

when cli  -yn -ys not effect ,AbstractYarnClusterDescriptor's 
validClusterSpecification is not final application resources from yarn  (cli  
-yn -ys not effect can refer to 
https://issues.apache.org/jira/browse/FLINK-11376)

 

the cli :

flink-1.7.1/bin/flink  run -m yarn-cluster -yn 1 -ys 1  -ynm test     
-ccom.ishansong.bigdata.Main222  ./flinkDemo-1.0-SNAPSHOT.jar

AbstractYarnClusterDescriptor's log :

 org.apache.flink.yarn.AbstractYarnClusterDescriptor           - Cluster 
specification: ClusterSpecification\{masterMemoryMB=1024, 
taskManagerMemoryMB=1024, numberTaskManagers=1, slotsPerTaskManager=1}

but yarn web ui:

!image-2019-01-17-14-57-24-060.png!



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)