My text logs don't have a stack trace with this exception. I'm doing this inside Flink SQL with a standard Kinesis connector and JSON formatter.
On Tue, Nov 29, 2022 at 6:38 PM yuxia <luoyu...@alumni.sjtu.edu.cn> wrote: > Which code line the error message happens? Maybe it will swallow the > exception and then log the error message, in which case Flink job won't > fail since it seems like no exception happens. > > Best regards, > Yuxia > > ------------------------------ > *发件人: *"Dan Hill" <quietgol...@gmail.com> > *收件人: *"User" <user@flink.apache.org> > *发送时间: *星期三, 2022年 11 月 30日 上午 8:06:52 > *主题: *Flink Table Kinesis sink not failing when sink fails > > I set up a simple Flink SQL job (Flink v1.14.4) that writes to Kinesis. > The job looks healthy but the records are not being written. I did not > give enough IAM permissions to write to Kinesis. However, the Flink SQL > job acts like it's healthy and checkpoints even though the Kinesis PutRecords > call fails. I'd expect this error to kill the Flink job. > I looked through Flink Jira and the Flink user group but didn't see a > similar issue. > > Is the silent failure a known issue? If the Flink job doesn't fail, it'll > be hard to detect production issues. > > ``` > > 2022-11-29 23:30:27,587 ERROR > org.apache.flink.kinesis.shaded.com.amazonaws.services.kinesis.producer.LogInputStreamReader > [] - [2022-11-29 23:30:27.578072] [0x00001e3b][0x00007f12ef8fc700] [error] > [AWS Log: ERROR](AWSClient)HTTP response code: 400 > Exception name: AccessDeniedException > Error message: User: > arn:aws:sts::055315558257:assumed-role/dev-workers-us-east-1b-2022031014331389150000000a/i-09e4f747a4bdbb1f0 > is not authorized to perform: kinesis:ListShards on resource: > arn:aws:kinesis:us-east-1:055315558257:stream/dan-dev-content-metrics because > no identity-based policy allows the kinesis:ListShards action > 6 response headers: > connection : close > content-length : 379 > content-type : application/x-amz-json-1.1 > date : Tue, 29 Nov 2022 23:30:27 GMT > x-amz-id-2 : > q8kuplUOMJILzVU97YA+TYSyy6aozeoST+yws26rOkyzEUUZT0zKBdcMWUAjV/8RrnMeed/+em7CbjpwzGYEANgkwCihZWdC > x-amzn-requestid : e4a39674-66fa-4dcd-b8a3-0e273e5e628a > > ``` > >
