well the 405 is due to a syntax error in the connector.json. after fixing
that, passing the -k switch, it started. but when looking at the
connect.log, mm2 is only talking in plaintext rather than SSL. After a
while it timed out because the port 9094 is ssl while mm2 is trying to use
plaintext. at least got past the curl problem.
On Wed, Apr 14, 2021 at 12:11 PM Men Lim <zulu...@gmail.com> wrote:
> I read thru the security_ssl page when I started this, it doesn't apply
> much to me because I'm running this in AWS MSK, where I can't access the
> broker. so my hands are tied there when it come to certificate.
>
> however, this morning, I decided to work on creating a self sign cert for
> the CURL command. I was able to get past the ssl handshake error and now
> running into another issue.
>
> * Trying 127.0.0.1...
> * TCP_NODELAY set
> * Connected to localhost (127.0.0.1) port 8443 (#0)
> * ALPN, offering h2
> * ALPN, offering http/1.1
> * Cipher selection:
> ALL:!EXPORT:!EXPORT40:!EXPORT56:!aNULL:!LOW:!RC4:@STRENGTH
> * successfully set certificate verify locations:
> * CAfile: /etc/pki/tls/certs/ca-bundle.crt
> CApath: none
> * TLSv1.2 (OUT), TLS header, Certificate Status (22):
> * TLSv1.2 (OUT), TLS handshake, Client hello (1):
> * TLSv1.2 (IN), TLS handshake, Server hello (2):
> * TLSv1.2 (IN), TLS handshake, Certificate (11):
> * TLSv1.2 (OUT), TLS alert, unknown CA (560):
>
> ** SSL certificate problem: self signed certificate* Closing connection 0*
>
> so I tried passing the -k flag and got the following message
>
> * Trying 127.0.0.1...
> * TCP_NODELAY set
> * Connected to localhost (127.0.0.1) port 8443 (#0)
> * ALPN, offering h2
> * ALPN, offering http/1.1
> * Cipher selection:
> ALL:!EXPORT:!EXPORT40:!EXPORT56:!aNULL:!LOW:!RC4:@STRENGTH
> * successfully set certificate verify locations:
> * CAfile: /etc/pki/tls/certs/ca-bundle.crt
> CApath: none
> * TLSv1.2 (OUT), TLS header, Certificate Status (22):
> * TLSv1.2 (OUT), TLS handshake, Client hello (1):
> * TLSv1.2 (IN), TLS handshake, Server hello (2):
> * TLSv1.2 (IN), TLS handshake, Certificate (11):
> * TLSv1.2 (IN), TLS handshake, Server key exchange (12):
> * TLSv1.2 (IN), TLS handshake, Server finished (14):
> * TLSv1.2 (OUT), TLS handshake, Client key exchange (16):
> * TLSv1.2 (OUT), TLS change cipher, Change cipher spec (1):
> * TLSv1.2 (OUT), TLS handshake, Finished (20):
> * TLSv1.2 (IN), TLS change cipher, Change cipher spec (1):
> * TLSv1.2 (IN), TLS handshake, Finished (20):
> * SSL connection using TLSv1.2 / ECDHE-RSA-AES256-GCM-SHA384
> * ALPN, server did not agree to a protocol
> * Server certificate:
> * subject: C=us; ST=il; L=Unknown; O=usfoods; OU=Unknown; CN=Unknown
> * start date: Apr 14 16:52:07 2021 GMT
> * expire date: Apr 12 16:52:07 2031 GMT
> * issuer: C=US; ST=MI; L=Unknown; O=FUD; OU=Unknown; CN=Unknown
> * SSL certificate verify result: self signed certificate (18), continuing
> anyway.
> > PUT /connectors HTTP/1.1
> > Host: localhost:8443
> > User-Agent: curl/7.61.1
> > Accept: */*
> > Content-Type: application/json
> > Content-Length: 1251
> > Expect: 100-continue
> >
> < HTTP/1.1 100 Continue
> * We are completely uploaded and fine
> < HTTP/1.1 405 Method Not Allowed
> < Date: Wed, 14 Apr 2021 19:07:01 GMT
> < Content-Length: 58
> < Server: Jetty(9.4.33.v20201020)
> <
> * Connection #0 to host localhost left intact
> *{"error_code":405,"message":"HTTP 405 Method Not Allowed"}*
>
> with the error, the mm2 process still isn't running. i'm not sure how to
> go about troubleshooting this HTTP 405 msg. Maybe the cert can't be self
> signed and need an official one.
>
>
> On Tue, Apr 13, 2021 at 8:41 PM Ning Zhang <ning2008w...@gmail.com> wrote:
>
>> assume your target / destination kafka cluster is SSL enabled. If your
>> MM2 wants to write to such cluster, you may have the following config in
>> your MM2:
>>
>>
>> https://github.com/ning2008wisc/minikube-mm2-demo/blob/master/kafka-mm/values.yaml#L79-L80
>>
>> on the broker (even client) side, you may refer to:
>>
>> http://kafka.apache.org/090/documentation.html#security_ssl
>>
>> On 2021/04/09 15:01:26, Men Lim <zulu...@gmail.com> wrote:
>> > Hi Ning,
>> >
>> > thanks for the response. This self sign cert stays on the ec2 instance,
>> > specifically for the curl command and I don't have to share it with the
>> > brokers correct?
>> >
>> > thanks,
>> >
>> >
>> >
>> > On Fri, Apr 9, 2021 at 7:55 AM Ning Zhang <ning2008w...@gmail.com>
>> wrote:
>> >
>> > > Hi Men,
>> > >
>> > > I used to deploy MM2 on EC2 with SSL and IIRC, probably give a try of
>> > > self-signing certs and key for testing purpose:
>> > > https://linuxize.com/post/creating-a-self-signed-ssl-certificate/
>> > >
>> > > On 2021/04/09 03:14:30, Men Lim <zulu...@gmail.com> wrote:
>> > > > Hi Ryanne,
>> > > >
>> > > > thanks for the reply. My kafka clusters are on AWS, their
>> serverless
>> > > > platform, MSK. I'm stuck with using the default java cacerts
>> unless I
>> > > use
>> > > > their AWS PCA which is pretty pricey.
>> > > >
>> > > > I ran the CURL command yesterday with the -v and --tlsv1.2 flag and
>> got
>> > > the
>> > > > following verbose message:
>> > > >
>> > > > curl -s -X POST -H 'Content-Type: application/json' --data
>> > > @connector.json
>> > > > https://localhost:8443/connectors -v --tlsv1.2
>> > > > * Trying 127.0.0.1...
>> > > > * TCP_NODELAY set
>> > > > * Connected to localhost (127.0.0.1) port 8443 (#0)
>> > > > * ALPN, offering h2
>> > > > * ALPN, offering http/1.1
>> > > > * Cipher selection:
>> > > > ALL:!EXPORT:!EXPORT40:!EXPORT56:!aNULL:!LOW:!RC4:@STRENGTH
>> > > > * successfully set certificate verify locations:
>> > > > * CAfile: /etc/pki/tls/certs/ca-bundle.crt
>> > > > CApath: none
>> > > > * TLSv1.2 (OUT), TLS header, Certificate Status (22):
>> > > > * TLSv1.2 (OUT), TLS handshake, Client hello (1):
>> > > > * TLSv1.2 (IN), TLS header, Unknown (21):
>> > > > * TLSv1.2 (IN), TLS alert, handshake failure (552):
>> > > > * error:14077410:SSL routines:SSL23_GET_SERVER_HELLO:sslv3 alert
>> > > handshake
>> > > > failure
>> > > >
>> > > > Thanks
>> > > >
>> > > > On Mon, Apr 5, 2021 at 7:26 AM Ryanne Dolan <ryannedo...@gmail.com>
>> > > wrote:
>> > > >
>> > > > > Yes it's possible. The most common issue in my experience is the
>> > > location
>> > > > > of the trust store and key store being different or absent on some
>> > > hosts.
>> > > > > You need to make sure that these locations are consistent across
>> all
>> > > hosts
>> > > > > in your Connect cluster, or use a ConfigProvider to provide the
>> > > location
>> > > > > dynamically. Otherwise, a task will get scheduled on some host and
>> > > fail to
>> > > > > find these files.
>> > > > >
>> > > > > Ryanne
>> > > > >
>> > > > >
>> > > > > On Wed, Mar 31, 2021, 8:22 PM Men Lim <zulu...@gmail.com> wrote:
>> > > > >
>> > > > > > Hello. I was wondering if someone can help answer my
>> question. I'm
>> > > > > trying
>> > > > > > to run MirrorMaker 2 in distributed mode using SSL. I have the
>> > > > > distributor
>> > > > > > running in SSL but when I can't get the curl REST api to do so.
>> I saw
>> > > > > that
>> > > > > > kif-208 fixed this but I can't seem to implement it.
>> > > > > >
>> > > > > > in my mm2-dist.prop file I have set:
>> > > > > > ////
>> > > > > > listeners=https://localhost:8443
>> > > > > > security.protocol=SSL
>> > > > > >
>> > > > > >
>> > > > >
>> > >
>> ssl.truststore.location=/home/ec2-user/kafka_2.13-2.7.0/cert/kafka.client.truststore.jks
>> > > > > > ////
>> > > > > > my connector.json file look like this:
>> > > > > >
>> > > > > > ////
>> > > > > > {
>> > > > > > "name": "mm2-connect-cluster",
>> > > > > > "config":{
>> > > > > > "connector.class":
>> > > > > "org.apache.kafka.connect.mirror.MirrorSourceConnector",
>> > > > > > "connector.client.config.override.policy": "All",
>> > > > > > "name": "mm2-connect-cluster",
>> > > > > > "topics": "test.*",
>> > > > > > "tasks.max": "1",
>> > > > > > "source.cluster.alias": "source",
>> > > > > > "target.cluster.alias": "target",
>> > > > > > "source.cluster.bootstrap.servers": "source:9094",
>> > > > > > "target.cluster.bootstrap.servers": "target:9094",
>> > > > > > "source->target.enabled": "true",
>> > > > > > "target->source.enabled": "false",
>> > > > > > "offset-syncs.topic.replication.factor": "4",
>> > > > > > "topics.exclude": ".*[\\-\\.]internal, .*\\.replica,
>> > > > > > __consumer_offsets",
>> > > > > > "groups.blacklist": "console-consumer-.*, connect-.*,
>> __.*",
>> > > > > > "topic.creation.enabled": "true",
>> > > > > > "topic.creation.default.replication.factor": "4",
>> > > > > > "topic.creation.default.partitions": "1"
>> > > > > > "key.converter":
>> > > "org.apache.kafka.connect.json.JsonConverter",
>> > > > > > "value.converter":
>> > > "org.apache.kafka.connect.json.JsonConverter",
>> > > > > > "security.protocol": "SSL",
>> > > > > > "ssl.truststore.password":
>> > > > > >
>> "/home/ec2-user/kafka_2.13-2.7.0/cert/kafka.client.truststore.jks"
>> > > > > > }
>> > > > > > }
>> > > > > > ////
>> > > > > >
>> > > > > > I would then start up the distributor and it launched fine. So
>> I
>> > > try to
>> > > > > > run the CURl command
>> > > > > >
>> > > > > > ////
>> > > > > > curl -s -X POST -H 'Content-Type: application/json' --data
>> > > > > @connector.json
>> > > > > > https://localhost:8443/connectors
>> > > > > > ////
>> > > > > > nada. nothing. no error. no reasons for not starting.
>> > > > > >
>> > > > > > Is it possible to run MM2 with SSL? If so, can someone point
>> me to a
>> > > > > > working example?
>> > > > > >
>> > > > > > thanks.
>> > > > > >
>> > > > >
>> > > >
>> > >
>> >
>>
>
