Has there been any news?
I'd love to get this tied up so that you're able to proceed.
Karl
On Thu, Jan 16, 2020 at 12:08 PM Jörn Franke <jornfra...@gmail.com> wrote:
> Ok I understand. I will try and let you know. Thanks again very much for
> your fast and detailed answer. Really appreciated. I hope I can give back
> with the solution to fetch WSDLs from https and maybe a solution to this
> problem (maybe other will face this as well).
>
> About the connector: the WSDL is successfully fetched via https (not file
> - no clue why) - after the modification I made. The only problem I see now
> is that the xsd to which the WSDL is referring are not fetched. The bizarre
> thing is that the https url that it mention for the xsd is absolutely
> correct. So I assume it does not understand an http url, maybe that is
> related to configuration.
>
> Am 16.01.2020 um 14:53 schrieb Karl Wright <daddy...@gmail.com>:
>
>
> The WSDLS are bundled with the jar. We intended this to be the ONLY way
> the wsdls were accessed, and made lots of changes to the wsdls accordingly,
> so that they referenced other wsdls via the "file system". The wsdls are
> the fixed up ones that are used to build the java stubs locally, plus a
> config file that's supposed to tell CXF how to resolve referenced wsdls.
> That config file may or may not be correct, because we never were able to
> get CXF to use the local resource wsdls during actual connection.
>
> Except now they seem to be both fetched via https AND locally sourced. I
> have no idea how that can be. I had assumed it was done one way or the
> other but not both.
>
> Perhaps the problem is that the configuration file is being read but the
> resource wsdls are not being found? Removing the meta-inf from the jar
> would then force everything to go through https. Ideally I'd love it if
> that wasn't needed and we could get the resource fetch working everywhere.
>
>
> Karl
>
>
> On Thu, Jan 16, 2020 at 8:20 AM Jörn Franke <jornfra...@gmail.com> wrote:
>
>> Well i am not sure how they solved it - I will share a tested solution in
>> Jira and everyone can check. Maybe their wsdl is accessible through http?
>>
>> What works is doing call through https, but thee fetching of WSDL did
>> not - as this is through another mechanism.
>>
>> I don’t think that the open text is different, the WSDL look very similar
>> to the repository.
>>
>> The strange thing is that for this error message it tries to access the
>> xsd through a https url (which is perfectly accessible for the server).
>> Could it be that the connector restrict itself somehow to local file
>> system only or similar?
>> Have you faced this issue before?
>>
>>
>>
>> Am 16.01.2020 um 12:56 schrieb Karl Wright <daddy...@gmail.com>:
>>
>>
>> I should say that we have (AFAICT) at least two independent installations
>> of the csws connector working in the field, at least one of them using
>> secure connections.
>>
>> Karl
>>
>>
>> On Thu, Jan 16, 2020 at 6:54 AM Karl Wright <daddy...@gmail.com> wrote:
>>
>>> We solved the WSDL fetching through HTTPS, or thought we had, by
>>> restructuring the code according to a number of articles we found. This
>>> was supposedly tested and worked in one installation. Nobody has ever
>>> reported issues with the wsdls being fetched however; I worry that you may
>>> have a different version of OpenText that is incompatible with the one we
>>> developed against. That's the problem with this kind of architecture;
>>> unless the wsdls are included in the jar there can be issues. We tried to
>>> do that too but were unable to get it to work.
>>>
>>> Karl
>>>
>>>
>>> On Thu, Jan 16, 2020 at 5:34 AM Jörn Franke <jornfra...@gmail.com>
>>> wrote:
>>>
>>>> Ok i fixed the source to fetch WSDL from https (it is not perfect yet
>>>> as it does not use the truststore yet but this I can fix) - I will share
>>>> later in a Jira.
>>>> It is however now unable to locate the imported document
>>>> /Authentication?xsd=2 relative to Authenticaton?wsdl#types1
>>>>
>>>> I will look into this, but if someone has come cross it then please let
>>>> me know.
>>>>
>>>> Am 16.01.2020 um 10:22 schrieb Jörn Franke <jornfra...@gmail.com>:
>>>>
>>>>
>>>> Coming back to the original topic. I believe SSL was never fully solved
>>>> from what i read in the corresponding issue. Apparently, the fetching of
>>>> the WSDL itself through https was not possible. Do you remember still some
>>>> insights beyond what is written in the issue ?
>>>>
>>>> Am 16.01.2020 um 00:37 schrieb Karl Wright <daddy...@gmail.com>:
>>>>
>>>>
>>>> Let me think about that option.
>>>>
>>>> Karl
>>>>
>>>>
>>>> On Wed, Jan 15, 2020 at 5:38 PM Jörn Franke <jornfra...@gmail.com>
>>>> wrote:
>>>>
>>>>> We could make it configurable, e.g. in properties.xml. Here people
>>>>> could set it to SSL, TLS, TLSv1.2 (to restrict it to TLS1.2 => some
>>>>> companies may want that!). Is this a viable option? That would be also
>>>>> future proof. We can leave it by default to SSL, but we should put in the
>>>>> example config files TLS by default (so new starters do not get even the
>>>>> idea to use an outdated protocol) AND put a comment with recommendation to
>>>>> use/enforce always newest protocols for security reasons. Of course, the
>>>>> choice is then with the people using the software.
>>>>> Could that be something sensible from your point of view?
>>>>>
>>>>> On Wed, Jan 15, 2020 at 11:14 PM Karl Wright <daddy...@gmail.com>
>>>>> wrote:
>>>>>
>>>>>> It's rather immaterial what browsers do here. What's important is
>>>>>> what *existing servers* support, since that is what we're connecting
>>>>>> with.
>>>>>>
>>>>>> I tend to agree that *most* people have probably upgraded to web
>>>>>> servers that support TLS. But we can't guarantee it, nor can we assume
>>>>>> that people have upgraded to the most modern version of TLS exclusively.
>>>>>> In fact I think we can assume they have *not*. When the SSL issues were
>>>>>> discovered a couple of years back, the standard recommendation was simply
>>>>>> to *disable* SSLv1 and SSLv2, not to upgrade to Java 11 or some such. We
>>>>>> still support (and have people using!!) early forms of NTLM (v1 to be
>>>>>> specific), for instance. We're not going to be able to wag the dog here.
>>>>>> Breaking changes of this kind usually mean we go to a whole new major
>>>>>> version of MCF.
>>>>>>
>>>>>> However, if you can show that SSLContext.getSSLFactory("TLS")
>>>>>> produces a SSLSocketFactory that works with all versions of TLS and SSL
>>>>>> that do not have known security holes, I would support changing over to
>>>>>> that. If it turns out we need much more specificity about the kind of
>>>>>> SSLSocketFactory we produce, then we need a better solution anyhow for
>>>>>> handling multiple protocols in one socket factory.
>>>>>>
>>>>>> Karl
>>>>>>
>>>>>>
>>>>>> On Wed, Jan 15, 2020 at 5:17 AM Jörn Franke <jornfra...@gmail.com>
>>>>>> wrote:
>>>>>>
>>>>>>> Hi Karl,
>>>>>>>
>>>>>>> No it does not. I can look into that further, but Current browsers
>>>>>>> stop supporting anything below TLSv1.2 in March 2020.
>>>>>>> Then TLS exists since more than ten years. I expect any server
>>>>>>> running nowadays will always have tls support.
>>>>>>> SSL itself is not supported since some time now. From a security
>>>>>>> perspective it should even break servers that run only SSL as they are
>>>>>>> inherently insecure and also clients that only support SSL are adding to
>>>>>>> this.
>>>>>>> However if you have an idea how this should be made configurable
>>>>>>> then I can look into this.
>>>>>>>
>>>>>>> Best regards
>>>>>>>
>>>>>>> Am 15.01.2020 um 10:52 schrieb Karl Wright <daddy...@gmail.com>:
>>>>>>>
>>>>>>>
>>>>>>> Hi,
>>>>>>>
>>>>>>> Mcf currently requires jdk8. Jdk11 is non trivial to support
>>>>>>> because of the removal of many jdk classes connectors need. It will be
>>>>>>> ported at some point but not lightly.
>>>>>>>
>>>>>>> Similarly, disabling SSL would certainly break many installations
>>>>>>> upon upgrade and we do not do that lightly.
>>>>>>>
>>>>>>> The core methods that mcf supplies its connectors should therefore
>>>>>>> be updated to support but not mandate tls. The protocol specification
>>>>>>> one
>>>>>>> gives to sslcontext is not a detailed one but rather a major version.
>>>>>>> What
>>>>>>> I don't know is whether"tlsv1" also allows for older protocols etc.
>>>>>>>
>>>>>>> Karl
>>>>>>>
>>>>>>> On Wed, Jan 15, 2020, 1:19 AM Jörn Franke <jornfra...@gmail.com>
>>>>>>> wrote:
>>>>>>>
>>>>>>>> Yes I am doing that but I will need to rebuild.
>>>>>>>> I don’t recommend TLSv1 - this is already outphased and will lock
>>>>>>>> out TLSv1.2. I try TLS only as it includes all TLS protocols (depends
>>>>>>>> on
>>>>>>>> JDK).
>>>>>>>>
>>>>>>>> SSL will not be supported by this (however as I said there are
>>>>>>>> other parts of the code where there is a getInstance(TLS). And some
>>>>>>>> caveats: On JDK6+7 TLS only means TLSv1 (and newer TLS Protocols are
>>>>>>>> deactivated) on JDK8 it means also that newer TLS protocols are
>>>>>>>> enabled.
>>>>>>>> To be honest in my opinion - a SSL only one is a significant
>>>>>>>> security hole and given how old TLS support is JDK i would be
>>>>>>>> surprised if
>>>>>>>> there is someone using such a server (most Organisations should switch
>>>>>>>> to
>>>>>>>> TLSv1.2 in any case as all protocols below have been broken).
>>>>>>>> While it works for all JDKs - probably JDK8 should be recommended
>>>>>>>> as it seems to have all TLS protocols activated when using „TLS“. Older
>>>>>>>> JDKs seem to deactivate TLSv1.1 and TLSv1.2 when using TLS. I will
>>>>>>>> write
>>>>>>>> more about this in the JIRA, once I verified that this solves the
>>>>>>>> problem.
>>>>>>>> Then TLSv1.3 is JDK11 only - I will investigate what that implies.
>>>>>>>> Does ManifoldCf supports JDK11?
>>>>>>>>
>>>>>>>> Am 15.01.2020 um 00:08 schrieb Karl Wright <daddy...@gmail.com>:
>>>>>>>>
>>>>>>>>
>>>>>>>> I think you can just change the code to read as follows when it
>>>>>>>> creates the SSLContext:
>>>>>>>>
>>>>>>>> SSLContext ctx = SSLContext.getInstance("TLSv1");
>>>>>>>> I don't know if TLS will downgrade to SSL if that's all that's
>>>>>>>> available.
>>>>>>>>
>>>>>>>>
>>>>>>>> Karl
>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>> On Tue, Jan 14, 2020 at 6:02 PM Jörn Franke <jornfra...@gmail.com>
>>>>>>>> wrote:
>>>>>>>>
>>>>>>>>> Yes it you do not change this setting as what I suspect happens
>>>>>>>>> here. See my previous mail for details.
>>>>>>>>>
>>>>>>>>> Am 14.01.2020 um 23:51 schrieb Karl Wright <daddy...@gmail.com>:
>>>>>>>>>
>>>>>>>>>
>>>>>>>>> It looks looks TLS is actually enabled in the SSLSocketFactory
>>>>>>>>> framework based on how you create the SSLSocketContext. See:
>>>>>>>>>
>>>>>>>>>
>>>>>>>>> https://docs.oracle.com/cd/E19698-01/816-7609/security-83/index.html
>>>>>>>>>
>>>>>>>>>
>>>>>>>>> Karl
>>>>>>>>>
>>>>>>>>>
>>>>>>>>> On Tue, Jan 14, 2020 at 5:48 PM Karl Wright <daddy...@gmail.com>
>>>>>>>>> wrote:
>>>>>>>>>
>>>>>>>>>> The design of ManifoldCF deliberately manages keystores on a
>>>>>>>>>> connection by connection basis, not globally. If you think the only
>>>>>>>>>> way to
>>>>>>>>>> implement TLS is via global keystore I very much doubt it.
>>>>>>>>>>
>>>>>>>>>> I am on the road until late tomorrow but somewhere along the line
>>>>>>>>>> I can do some research into why TLS won't work as we are currently
>>>>>>>>>> doing it.
>>>>>>>>>>
>>>>>>>>>> Karl
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>> On Tue, Jan 14, 2020 at 12:56 PM Jörn Franke <
>>>>>>>>>> jornfra...@gmail.com> wrote:
>>>>>>>>>>
>>>>>>>>>>> These are TLS only. So maybe you have other servers where tls
>>>>>>>>>>> and ssl are possible and it downgrades to ssl.however, this is
>>>>>>>>>>> speculation
>>>>>>>>>>> and I need to verify it. I have to rebuilt manifold for that.
>>>>>>>>>>> Probably I
>>>>>>>>>>> have to reinstall everything as the keystorefactory is a dependency
>>>>>>>>>>> in the
>>>>>>>>>>> connector.
>>>>>>>>>>>
>>>>>>>>>>> Am 14.01.2020 um 18:34 schrieb Karl Wright <daddy...@gmail.com>:
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>> If you can recommend changes to support TLS, that would be
>>>>>>>>>>> great. The basic infrastructure should still work; it is just a
>>>>>>>>>>> custom
>>>>>>>>>>> keystone and associated SSLSocketFactory, which I think also is
>>>>>>>>>>> used for
>>>>>>>>>>> TLS connections, unless I am missing something.
>>>>>>>>>>>
>>>>>>>>>>> On Tue, Jan 14, 2020, 9:38 AM Jörn Franke <jornfra...@gmail.com>
>>>>>>>>>>> wrote:
>>>>>>>>>>>
>>>>>>>>>>>> Yes this works fine. I believe the error comes from the fact
>>>>>>>>>>>> that TLS connections are not supported.
>>>>>>>>>>>>
>>>>>>>>>>>> Am 14.01.2020 um 15:31 schrieb Michael Cizmar <
>>>>>>>>>>>> michael.ciz...@mcplusa.com>:
>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>> If you want to test the url and the ssl, I would recommend
>>>>>>>>>>>> attempting using SSLPoke to confirm that they keystore is setup
>>>>>>>>>>>> properly:
>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>> https://github.com/MichalHecko/SSLPoke
>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>> Michael
>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>> *From: *Karl Wright <daddy...@gmail.com>
>>>>>>>>>>>> *Reply-To: *"user@manifoldcf.apache.org" <
>>>>>>>>>>>> user@manifoldcf.apache.org>
>>>>>>>>>>>> *Date: *Tuesday, January 14, 2020 at 7:21 AM
>>>>>>>>>>>> *To: *"user@manifoldcf.apache.org" <user@manifoldcf.apache.org>
>>>>>>>>>>>> *Subject: *Re: CSWS Connector : ServiceConstructionException:
>>>>>>>>>>>> Failed to create service
>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>> Hmm, others have succeeded setting up SSL connections with the
>>>>>>>>>>>> current code. Hoping they chime in here.
>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>> Karl
>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>> On Tue, Jan 14, 2020, 8:19 AM Jörn Franke <jornfra...@gmail.com>
>>>>>>>>>>>> wrote:
>>>>>>>>>>>>
>>>>>>>>>>>> It seems that it has indeed a certificate issue as it cannot
>>>>>>>>>>>> find a valid certification path to the target. The thing is: I
>>>>>>>>>>>> added those
>>>>>>>>>>>> certificates in the UI should it should not happen.
>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>> Am 10.01.2020 um 20:51 schrieb Jörn Franke <
>>>>>>>>>>>> jornfra...@gmail.com>:
>>>>>>>>>>>>
>>>>>>>>>>>> 2.15 ...
>>>>>>>>>>>>
>>>>>>>>>>>> I will try on the weekend to see if I can get some logs out of
>>>>>>>>>>>> it.
>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>> Am 10.01.2020 um 19:02 schrieb Karl Wright <daddy...@gmail.com
>>>>>>>>>>>> >:
>>>>>>>>>>>>
>>>>>>>>>>>> Can I ask what version of MCF you are using? There were issues
>>>>>>>>>>>> with SSL in the first release of the csws connector if I recall
>>>>>>>>>>>> correctly,
>>>>>>>>>>>> that were fixed for the second release.
>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>> Karl
>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>> On Fri, Jan 10, 2020 at 11:42 AM Jörn Franke <
>>>>>>>>>>>> jornfra...@gmail.com> wrote:
>>>>>>>>>>>>
>>>>>>>>>>>> I added root, intermediate and server certificate (in base64
>>>>>>>>>>>> cer, it seems to be recognized by manifoldcf), but I still get the
>>>>>>>>>>>> same
>>>>>>>>>>>> message. I will try to get somehow the full stacktrace
>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>> Am 10.01.2020 um 17:21 schrieb Karl Wright <daddy...@gmail.com
>>>>>>>>>>>> >:
>>>>>>>>>>>>
>>>>>>>>>>>> If you are using SSL you need to have the proper certificate
>>>>>>>>>>>> saved in the connection's keystore.
>>>>>>>>>>>>
>>>>>>>>>>>> Karl
>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>> On Fri, Jan 10, 2020 at 11:20 AM Jörn Franke <
>>>>>>>>>>>> jornfra...@gmail.com> wrote:
>>>>>>>>>>>>
>>>>>>>>>>>> It is actually a server using configuration of the command -
>>>>>>>>>>>> driven multi-process model (but the agents executed as a service
>>>>>>>>>>>> and the
>>>>>>>>>>>> war on a tomcat executed as a service) under Linux.
>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>> I thought as well that it cannot reach the webservices, the
>>>>>>>>>>>> question is why. On the same server I can reach the webservices
>>>>>>>>>>>> and fetch
>>>>>>>>>>>> the WSDL without issues.
>>>>>>>>>>>>
>>>>>>>>>>>> Maybe sth related to ssl ?
>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>> Am 10.01.2020 um 14:59 schrieb Karl Wright <daddy...@gmail.com
>>>>>>>>>>>> >:
>>>>>>>>>>>>
>>>>>>>>>>>> How are you running manifoldcf? Single process example, or a
>>>>>>>>>>>> custom setup of some kind?
>>>>>>>>>>>>
>>>>>>>>>>>> This exception is a "catch all" exception generated far below
>>>>>>>>>>>> anything in ManifoldCF, but usually means it cannot download the
>>>>>>>>>>>> WSDLs from
>>>>>>>>>>>> the service. Getting the full exception dumped in the log
>>>>>>>>>>>> requires a
>>>>>>>>>>>> "hack" to the check() method of the connector, but I'm pretty sure
>>>>>>>>>>>> that's
>>>>>>>>>>>> what's happening anyway.
>>>>>>>>>>>>
>>>>>>>>>>>> Karl
>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>> On Fri, Jan 10, 2020 at 8:50 AM Jörn Franke <
>>>>>>>>>>>> jornfra...@gmail.com> wrote:
>>>>>>>>>>>>
>>>>>>>>>>>> Hi,
>>>>>>>>>>>>
>>>>>>>>>>>> I tried to use the CSWS connector, but already for the
>>>>>>>>>>>> Authority connection I receive a
>>>>>>>>>>>> org.apache.cxf.service.factory.ServiceConstructionException:
>>>>>>>>>>>> Failed to
>>>>>>>>>>>> create service.
>>>>>>>>>>>>
>>>>>>>>>>>> Unfortunately I don’t see more details , also not in the log
>>>>>>>>>>>> (debug is activated). I try to get a little bit more output by
>>>>>>>>>>>> modifying
>>>>>>>>>>>> the connector, but maybe someone has already an idea why this can
>>>>>>>>>>>> happen?
>>>>>>>>>>>>
>>>>>>>>>>>> Are there some special instructions to use it? The pointers to
>>>>>>>>>>>> the webservices are correct, I tested via Curl and SOAPUI.
>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>> Thank you.
>>>>>>>>>>>> Best regards
>>>>>>>>>>>>
>>>>>>>>>>>>
