It seems that there might be some modifications to fuseki's shiro.ini file that could help things, but I'm just not sure.
J On Thu, Aug 27, 2015 at 10:56 AM, Jason Levitt <slimands...@gmail.com> wrote: > If I remove that line from my config file: > > <Set name="host">mysite.com</Set> > > And then run fuseki and try to connect, using openssl, I get: > > $ openssl s_client -connect mysite.com:8443 > CONNECTED(00000003) > 5546:error:140790E5:SSL routines:SSL23_WRITE:ssl handshake > failure:/SourceCache/OpenSSL098/OpenSSL098-52.40.1/src/ssl/s23_lib.c:185: > > I have no idea what this means -- probably a certificate mismatch(?). > > J > > On Thu, Aug 27, 2015 at 10:46 AM, Andy Seaborne <a...@apache.org> wrote: >> On 27/08/15 15:20, Jason Levitt wrote: >>> >>> Hi Andy, >>> >>> Not only is Jetty 9.1 fairly different than Jetty 8, but the current >>> version of Jetty, 9.3 is somewhat different >>> than 9.1. I will investigate further. >> >> >> I just tried out 9.3 by flipping the version to 9.3.2.v20150730 and it seem >> OK (after 5 mins playing with it....). So looks like the codebase can >> switch if that helps simplifies things. >> >> Andy >> >> >> >>> >>> J >>> >>> On Thu, Aug 27, 2015 at 6:42 AM, Andy Seaborne <a...@apache.org> wrote: >>>> >>>> Jason - thank you for pushing on with this. It seems a lot of Jetty has >>>> changed Jetty8->Jetty9.1 in this area which is all news to me. >>>> >>>> On 27/08/15 06:09, Jason Levitt wrote: >>>>> >>>>> >>>>> Making some progress but things still don't work. >>>>> >>>>> The startup log (edited) looks like this (domain name changed to >>>>> mysite.com): >>>> >>>> >>>> >>>> This looks like it is because its asking to run on an address that isn't >>>> the >>>> local machine for some reason. It does not look like something on the >>>> same >>>> port because it is (usually) >>>> >>>> "java.net.BindException: Address already in use" >>>> >>>> but it's might be worth checking. On Linux, "sudo lsof -i:8443" >>>> >>>> Your config does not set the host but maybe the IP config is getting in >>>> the >>>> way. This is EC2 so does the real DNS name resolve to the IP address of a >>>> local interface? Does using "localhost" work (= do something different)? >>>> >>>> That's the best clue I could find on StackOverflow. I haven't found a >>>> way >>>> to get the same error message using plan HTTP on a non-EC2 machine >>>> though. >>>> >>>>> [2015-08-27 03:56:03] Server ERROR SPARQLServer (port=0): Failed >>>>> to start server: Cannot assign requested address >>>> >>>> >>>> port=0 looks weird though if you are taking control with the config file >>>> that is possible due to the earlier error. >>>> >>>> What is printed is serverConnector.getPort() and serverConnector is the >>>> first/only configured ServerConnector. >>>> >>>> Andy >>>> >>>> >>>> >>>>> >>>>> [2015-08-27 03:56:03] Server INFO Jetty server config file = >>>>> myconfig.xml >>>>> [2015-08-27 03:56:03] Server INFO Fuseki 2.3.0 >>>>> 2015-07-25T17:11:28+0000 >>>>> [2015-08-27 03:56:03] Config INFO FUSEKI_HOME=/home/ec2-user/fuseki >>>>> [2015-08-27 03:56:03] Config INFO >>>>> FUSEKI_BASE=/home/ec2-user/fuseki/run >>>>> [2015-08-27 03:56:03] Servlet INFO Initializing Shiro environment >>>>> [2015-08-27 03:56:03] Config INFO Shiro file: >>>>> file:///home/ec2-user/fuseki/run/shiro.ini >>>>> [2015-08-27 03:56:03] Config INFO Template file: >>>>> templates/config-tdb-dir >>>>> [2015-08-27 03:56:03] Config INFO TDB dataset: directory=ds >>>>> [2015-08-27 03:56:03] Config INFO Register: /ds >>>>> [2015-08-27 03:56:03] AbstractLifeCycle WARN FAILED >>>>> ServerConnector@7e5441{SSL-http/1.1}{mysite.com:8443}: >>>>> java.net.BindException: Cannot assign requested address >>>>> java.net.BindException: Cannot assign requested address >>>>> at sun.nio.ch.Net.bind0(Native Method) >>>>> at sun.nio.ch.Net.bind(Net.java:433) >>>>> at sun.nio.ch.Net.bind(Net.java:425) >>>>> .... >>>>> .... >>>>> .... >>>>> [2015-08-27 03:56:03] AbstractLifeCycle WARN FAILED >>>>> org.eclipse.jetty.server.Server@f9ed3e: java.net.BindException: Cannot >>>>> assign requested address >>>>> java.net.BindException: Cannot assign requested address >>>>> at sun.nio.ch.Net.bind0(Native Method) >>>>> at sun.nio.ch.Net.bind(Net.java:433) >>>>> at sun.nio.ch.Net.bind(Net.java:425) >>>>> .... >>>>> .... >>>>> .... >>>>> [2015-08-27 03:56:03] Server ERROR SPARQLServer (port=0): Failed >>>>> to start server: Cannot assign requested address >>>>> >>>>> >>>>> And I'm running fuseki 2.3.0 with this command: >>>>> >>>>> nohup ./fuseki-server --port 8443 --update >>>>> --jetty-config=myconfig.xml --loc=ds /ds >>>>> >>>>> >>>>> The "myconfig.xml" file is below (I've already added my certificate >>>>> and key to the Java 8 JSSE): >>>>> >>>>> <?xml version="1.0"?> >>>>> <!DOCTYPE Configure PUBLIC "-//Jetty//Configure//EN" >>>>> "http://www.eclipse.org/jetty/configure.dtd"> >>>>> >>>>> <Configure id="Server" class="org.eclipse.jetty.server.Server"> >>>>> <Call name="addConnector"> >>>>> <Arg> >>>>> <New class="org.eclipse.jetty.server.ServerConnector"> >>>>> <Arg name="server"><Ref refid="Server" /></Arg> >>>>> <Arg name="factories"> >>>>> <Array type="org.eclipse.jetty.server.ConnectionFactory"> >>>>> <Item> >>>>> <New class="org.eclipse.jetty.server.HttpConnectionFactory"> >>>>> <Arg name="config"><Ref refid="httpConfig" /></Arg> >>>>> </New> >>>>> </Item> >>>>> </Array> >>>>> </Arg> >>>>> </New> >>>>> </Arg> >>>>> </Call> >>>>> >>>>> <New id="sslContextFactory" >>>>> class="org.eclipse.jetty.util.ssl.SslContextFactory"> >>>>> <Set name="KeyStorePath">/home/ec2-user/keystore</Set> >>>>> <Set name="KeyStorePassword">somepassword</Set> >>>>> <Set name="KeyManagerPassword">somepassword</Set> >>>>> <Set name="TrustStorePath">/home/ec2-user/keystore</Set> >>>>> <Set name="TrustStorePassword">somepassword</Set> >>>>> </New> >>>>> >>>>> <Call id="sslConnector" name="addConnector"> >>>>> <Arg> >>>>> <New class="org.eclipse.jetty.server.ServerConnector"> >>>>> <Arg name="server"><Ref refid="Server" /></Arg> >>>>> <Arg name="factories"> >>>>> <Array type="org.eclipse.jetty.server.ConnectionFactory"> >>>>> <Item> >>>>> <New >>>>> class="org.eclipse.jetty.server.SslConnectionFactory"> >>>>> <Arg name="next">http/1.1</Arg> >>>>> <Arg name="sslContextFactory"><Ref >>>>> refid="sslContextFactory"/></Arg> >>>>> </New> >>>>> </Item> >>>>> <Item> >>>>> <New >>>>> class="org.eclipse.jetty.server.HttpConnectionFactory"> >>>>> <Arg name="config"><Ref refid="tlsHttpConfig"/></Arg> >>>>> </New> >>>>> </Item> >>>>> </Array> >>>>> </Arg> >>>>> <Set name="host"><Property name="jetty.host"/></Set> >>>>> <Set name="port"><Property name="jetty.tls.port" >>>>> default="8443" >>>>> /></Set> >>>>> <Set name="idleTimeout">30000</Set> >>>>> <Set name="host">mysite.com</Set> >>>>> </New> >>>>> </Arg> >>>>> </Call> >>>>> >>>>> </Configure> >>>>> >>>>> =================================== >>>>> >>>>> On Tue, Aug 25, 2015 at 5:17 PM, Jason Levitt <slimands...@gmail.com> >>>>> wrote: >>>>>> >>>>>> >>>>>> I can't find any examples of the file that you hand to >>>>>> "--jetty-config" >>>>>> >>>>>> The "official" jetty docs for configuring SSL imply that there are two >>>>>> configuration files, >>>>>> jetty-ssl-context.xml and jetty-https.xml. >>>>>> >>>>>> (http://www.eclipse.org/jetty/documentation/current/configuring-ssl.html) >>>>>> >>>>>> The example that you cite: >>>>>> >>>>>> >>>>>> >>>>>> http://git.eclipse.org/c/jetty/org.eclipse.jetty.project.git/plain/jetty-server/src/main/config/etc/jetty-https.xml >>>>>> >>>>>> says that: "This configuration must be used in conjunction with >>>>>> jetty.xml and jetty-ssl.xml" >>>>>> >>>>>> Where do these files go? The Fuseki download does not have any "etc" >>>>>> directory or >>>>>> any xml configuration files at all. >>>>>> >>>>>> So, I tried handing some XML config files to Fuseki using >>>>>> --jetty-config and it gives very little >>>>>> info in the error (see below). Has anyone actually successfully run >>>>>> Fuseki over SSL? >>>>>> >>>>>> [2015-08-25 22:13:34] Server INFO Jetty server config file = >>>>>> ./jetty-https.xml >>>>>> [2015-08-25 22:13:34] Server ERROR SPARQLServer: Failed to >>>>>> configure server: Unknown configuration type: Call in >>>>>> org.eclipse.jetty.xml.XmlConfiguration@1d80d2b >>>>>> java.lang.IllegalStateException: Unknown configuration type: Call in >>>>>> org.eclipse.jetty.xml.XmlConfiguration@1d80d2b >>>>>> at >>>>>> >>>>>> org.eclipse.jetty.xml.XmlConfiguration.setConfig(XmlConfiguration.java:198) >>>>>> at >>>>>> >>>>>> org.eclipse.jetty.xml.XmlConfiguration.<init>(XmlConfiguration.java:177) >>>>>> at >>>>>> >>>>>> org.apache.jena.fuseki.jetty.JettyFuseki.configServer(JettyFuseki.java:264) >>>>>> at >>>>>> >>>>>> org.apache.jena.fuseki.jetty.JettyFuseki.buildServerWebapp(JettyFuseki.java:222) >>>>>> at org.apache.jena.fuseki.jetty.JettyFuseki.<init>(JettyFuseki.java:91) >>>>>> at >>>>>> >>>>>> org.apache.jena.fuseki.jetty.JettyFuseki.initializeServer(JettyFuseki.java:86) >>>>>> at >>>>>> >>>>>> org.apache.jena.fuseki.cmd.FusekiCmd$FusekiCmdInner.exec(FusekiCmd.java:335) >>>>>> at jena.cmd.CmdMain.mainMethod(CmdMain.java:93) >>>>>> at jena.cmd.CmdMain.mainRun(CmdMain.java:58) >>>>>> at jena.cmd.CmdMain.mainRun(CmdMain.java:45) >>>>>> at >>>>>> >>>>>> org.apache.jena.fuseki.cmd.FusekiCmd$FusekiCmdInner.innerMain(FusekiCmd.java:96) >>>>>> at org.apache.jena.fuseki.cmd.FusekiCmd.main(FusekiCmd.java:59) >>>>>> org.apache.jena.fuseki.FusekiException: Failed to configure a server >>>>>> using configuration file './jetty-https.xml' >>>>>> at >>>>>> >>>>>> org.apache.jena.fuseki.jetty.JettyFuseki.configServer(JettyFuseki.java:269) >>>>>> at >>>>>> >>>>>> org.apache.jena.fuseki.jetty.JettyFuseki.buildServerWebapp(JettyFuseki.java:222) >>>>>> at org.apache.jena.fuseki.jetty.JettyFuseki.<init>(JettyFuseki.java:91) >>>>>> at >>>>>> >>>>>> org.apache.jena.fuseki.jetty.JettyFuseki.initializeServer(JettyFuseki.java:86) >>>>>> at >>>>>> >>>>>> org.apache.jena.fuseki.cmd.FusekiCmd$FusekiCmdInner.exec(FusekiCmd.java:335) >>>>>> at jena.cmd.CmdMain.mainMethod(CmdMain.java:93) >>>>>> at jena.cmd.CmdMain.mainRun(CmdMain.java:58) >>>>>> at jena.cmd.CmdMain.mainRun(CmdMain.java:45) >>>>>> at >>>>>> >>>>>> org.apache.jena.fuseki.cmd.FusekiCmd$FusekiCmdInner.innerMain(FusekiCmd.java:96) >>>>>> at org.apache.jena.fuseki.cmd.FusekiCmd.main(FusekiCmd.java:59) >>>>>> >>>>>> J >>>>>> >>>>>> >>>>>> On Fri, Aug 21, 2015 at 4:14 AM, Andy Seaborne <a...@apache.org> wrote: >>>>>>> >>>>>>> >>>>>>> On 20/08/15 22:37, Jason Levitt wrote: >>>>>>>> >>>>>>>> >>>>>>>> >>>>>>>> Thanks. So I can still use the "--jetty-config" option with Fuseki >>>>>>>> v2.30 ? >>>>>>>> >>>>>>>> J >>>>>>> >>>>>>> >>>>>>> >>>>>>> >>>>>>> Yes, should work to pass in the file. There was a major jetty version >>>>>>> change (8 to 9) and what effect that has had on that option is unclear >>>>>>> to >>>>>>> me. Connector changed Jetty 8->9 >>>>>>> >>>>>>> >>>>>>> http://www.eclipse.org/jetty/documentation/current/configuring-ssl.html >>>>>>> >>>>>>> and their example: >>>>>>> >>>>>>> >>>>>>> >>>>>>> http://git.eclipse.org/c/jetty/org.eclipse.jetty.project.git/plain/jetty-server/src/main/config/etc/jetty-https.xml >>>>>>> >>>>>>> It would be good to add this to the distribution - if you or anyone >>>>>>> else >>>>>>> has >>>>>>> a working version, I'd be very grateful to get a copy. >>>>>>> >>>>>>> >>>>>>> Andy >>>>>>> >>>>>>>> >>>>>>>> On Thu, Aug 20, 2015 at 3:46 PM, Andy Seaborne <a...@apache.org> >>>>>>>> wrote: >>>>>>>>> >>>>>>>>> >>>>>>>>> >>>>>>>>> On 20/08/15 21:24, Jason Levitt wrote: >>>>>>>>>> >>>>>>>>>> >>>>>>>>>> >>>>>>>>>> >>>>>>>>>> Which version of Jetty does Fuseki 2.30 (the latest version) use? >>>>>>>>>> >>>>>>>>>> J >>>>>>>>> >>>>>>>>> >>>>>>>>> >>>>>>>>> >>>>>>>>> >>>>>>>>> >>>>>>>>> >>>>>>>>> >>>>>>>>> http://central.maven.org/maven2/org/apache/jena/jena-fuseki/2.3.0/jena-fuseki-2.3.0.pom >>>>>>>>> >>>>>>>>> ==> Jetty 9.1.1.v20140108 >>>>>>>>> >>>>>>>>> Andy >>>>>>>>> >>>>>>>>> >>>>>>>>>> >>>>>>>>>> On Thu, Aug 20, 2015 at 6:14 AM, Andy Seaborne <a...@apache.org> >>>>>>>>>> wrote: >>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>> The Jetty documentation is the best place to go for details of >>>>>>>>>>> setting >>>>>>>>>>> up >>>>>>>>>>> Jetty. >>>>>>>>>>> >>>>>>>>>>> Here's one in the examples/ area but as far as I can tell it's >>>>>>>>>>> more >>>>>>>>>>> int >>>>>>>>>>> he >>>>>>>>>>> category of "should work" (it is from Fuseki1 and that was a >>>>>>>>>>> different >>>>>>>>>>> version of Jetty) rather than tested. >>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>> https://github.com/apache/jena/blob/master/jena-fuseki2/examples/jetty-fuseki.xml >>>>>>>>>>> >>>>>>>>>>> If you, or anyone else, has a better example - please send it. >>>>>>>>>>> >>>>>>>>>>> Andy >>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>> On 20/08/15 02:54, Jason Levitt wrote: >>>>>>>>>>>> >>>>>>>>>>>> >>>>>>>>>>>> >>>>>>>>>>>> >>>>>>>>>>>> >>>>>>>>>>>> We're in an AWS environment using Fuseki 2 with built-in Jetty. >>>>>>>>>>>> It >>>>>>>>>>>> only talks to internal machines so there >>>>>>>>>>>> is no need to protect it from external exposure. So that means >>>>>>>>>>>> that >>>>>>>>>>>> the easiest way is to use the >>>>>>>>>>>> `--jetty-config` flag to setup HTTPS to Jetty? Are there any >>>>>>>>>>>> docs >>>>>>>>>>>> on >>>>>>>>>>>> what the options are for that >>>>>>>>>>>> config file (e.g. what goes into the config file)? >>>>>>>>>>>> >>>>>>>>>>>> J >>>>>>>>>>>> >>>>>>>>>>>> On Tue, Aug 18, 2015 at 3:21 PM, Andy Seaborne <a...@apache.org> >>>>>>>>>>>> wrote: >>>>>>>>>>>>> >>>>>>>>>>>>> >>>>>>>>>>>>> >>>>>>>>>>>>> >>>>>>>>>>>>> >>>>>>>>>>>>> Right. In a production environment, a reverse proxy is useful >>>>>>>>>>>>> for >>>>>>>>>>>>> several >>>>>>>>>>>>> things and while there is nothing that force a reverse proxy, >>>>>>>>>>>>> the >>>>>>>>>>>>> weight >>>>>>>>>>>>> of >>>>>>>>>>>>> features can mean it's a useful and flexible thing to put into a >>>>>>>>>>>>> production >>>>>>>>>>>>> system. >>>>>>>>>>>>> >>>>>>>>>>>>> 1/ Blocking undesirable clients >>>>>>>>>>>>> (manic crawlers, badly written PHP scripts) >>>>>>>>>>>>> 2/ more robust to DOS attacks (and accidental attacks) >>>>>>>>>>>>> Java web containers just aren't as good under silly load >>>>>>>>>>>>> conditions. >>>>>>>>>>>>> 3/ URL rewrite >>>>>>>>>>>>> E.g don't need /dataset/query - can be any URL you like. >>>>>>>>>>>>> 4/ Security >>>>>>>>>>>>> integrate with local systems; rich choice of controls. >>>>>>>>>>>>> Control who and what can update >>>>>>>>>>>>> No need to restart for shiro chnages. >>>>>>>>>>>>> 5/ Rate control (e.g. no more than N queries at a time) >>>>>>>>>>>>> 6/ https (can be expensive so a C-implementation can help) >>>>>>>>>>>>> 7/ Lots of add-ons and mods for all sorts of tasks. >>>>>>>>>>>>> 8/ Lots of Q&A on stackoverflow! >>>>>>>>>>>>> >>>>>>>>>>>>> Fuseki has "--localhost" to only talk to the machine's localhost >>>>>>>>>>>>> network >>>>>>>>>>>>> interface. In an environment like AWS, where port control is >>>>>>>>>>>>> easily, >>>>>>>>>>>>> it's >>>>>>>>>>>>> trivial to secure the Fuseki server to only talk to the local >>>>>>>>>>>>> reverse >>>>>>>>>>>>> proxy >>>>>>>>>>>>> by blocking all ports except (22 and) 80+443. >>>>>>>>>>>>> >>>>>>>>>>>>> Andy >>>>>>>>>>>>> >>>>>>>>>>>>> >>>>>>>>>>>>> On 18/08/15 20:21, A. Soroka wrote: >>>>>>>>>>>>>> >>>>>>>>>>>>>> >>>>>>>>>>>>>> >>>>>>>>>>>>>> >>>>>>>>>>>>>> >>>>>>>>>>>>>> >>>>>>>>>>>>>> I checked more carefully (should have done that before >>>>>>>>>>>>>> replying) >>>>>>>>>>>>>> and >>>>>>>>>>>>>> it >>>>>>>>>>>>>> seems that Fuseki 2 also offers the `--jetty-config` flag for >>>>>>>>>>>>>> using >>>>>>>>>>>>>> a >>>>>>>>>>>>>> Jetty >>>>>>>>>>>>>> configuration that supports HTTPS: >>>>>>>>>>>>>> >>>>>>>>>>>>>> --jetty-config=FILE Set up the server (not services) with a >>>>>>>>>>>>>> Jetty >>>>>>>>>>>>>> XML >>>>>>>>>>>>>> file >>>>>>>>>>>>>> >>>>>>>>>>>>>> --- >>>>>>>>>>>>>> A. Soroka >>>>>>>>>>>>>> The University of Virginia Library >>>>>>>>>>>>>> >>>>>>>>>>>>>> On Aug 18, 2015, at 10:34 AM, aj...@virginia.edu >>>>>>>>>>>>>> <aj...@email.virginia.edu> wrote: >>>>>>>>>>>>>> >>>>>>>>>>>>>>> Are you deploying Fuseki to your own servlet container (e.g. >>>>>>>>>>>>>>> Tomcat >>>>>>>>>>>>>>> or >>>>>>>>>>>>>>> Jetty) or using the server included with Fuseki and is it >>>>>>>>>>>>>>> Fuskei >>>>>>>>>>>>>>> 1 >>>>>>>>>>>>>>> or >>>>>>>>>>>>>>> 2? >>>>>>>>>>>>>>> >>>>>>>>>>>>>>> If the former, you will need to supply configuration specific >>>>>>>>>>>>>>> to >>>>>>>>>>>>>>> that >>>>>>>>>>>>>>> container. If the latter and it is Fuseki 1, there is a Stack >>>>>>>>>>>>>>> Overflow >>>>>>>>>>>>>>> answer for it: >>>>>>>>>>>>>>> >>>>>>>>>>>>>>> >>>>>>>>>>>>>>> >>>>>>>>>>>>>>> >>>>>>>>>>>>>>> >>>>>>>>>>>>>>> >>>>>>>>>>>>>>> >>>>>>>>>>>>>>> https://stackoverflow.com/questions/28310045/enable-https-ssl-on-fuseki-server >>>>>>>>>>>>>>> >>>>>>>>>>>>>>> but the links seems to be dead. The idea is to supply your own >>>>>>>>>>>>>>> Jetty >>>>>>>>>>>>>>> configuration (Jetty is the servlet container that the Fuseki >>>>>>>>>>>>>>> command >>>>>>>>>>>>>>> uses). >>>>>>>>>>>>>>> For Fuseki 2, I think it is still under development? You could >>>>>>>>>>>>>>> use >>>>>>>>>>>>>>> a >>>>>>>>>>>>>>> reverse >>>>>>>>>>>>>>> proxy in front of Fuseki, in that case. >>>>>>>>>>>>>>> >>>>>>>>>>>>>>> --- >>>>>>>>>>>>>>> A. Soroka >>>>>>>>>>>>>>> The University of Virginia Library >>>>>>>>>>>>>>> >>>>>>>>>>>>>>> On Aug 17, 2015, at 7:07 PM, Jason Levitt >>>>>>>>>>>>>>> <slimands...@gmail.com> >>>>>>>>>>>>>>> wrote: >>>>>>>>>>>>>>> >>>>>>>>>>>>>>>> Sorry if this is a FAQ, but I'm wondering if there are >>>>>>>>>>>>>>>> any guidelines online to setting up >>>>>>>>>>>>>>>> Fuseki for HTTPS access? >>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>> Jason >>>>>>>>>>>>>>> >>>>>>>>>>>>>>> >>>>>>>>>>>>>>> >>>>>>>>>>>>>>> >>>>>>>>>>>>>>> >>>>>>>>>>>>>>> >>>>>>>>>>>>>>> >>>>>>>>>>>>>> >>>>>>>>>>>>> >>>>>>>>>>> >>>>>>>>> >>>>>>> >>>> >>