[jira] [Commented] (CLOUDSTACK-9238) URL fields in database are to small. Cause malformed URLs

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/CLOUDSTACK-9238?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15304622#comment-15304622
 ] 

ASF GitHub Bot commented on CLOUDSTACK-9238:


Github user asfgit closed the pull request at:

https://github.com/apache/cloudstack/pull/1567


> URL fields in database are to small. Cause malformed URLs
> -
>
> Key: CLOUDSTACK-9238
> URL: https://issues.apache.org/jira/browse/CLOUDSTACK-9238
> Project: CloudStack
>  Issue Type: Bug
>  Security Level: Public(Anyone can view this level - this is the 
> default.) 
>  Components: Management Server
>Affects Versions: 4.6.0, 4.7.0, 4.8.0
>Reporter: Wido den Hollander
>Assignee: Wido den Hollander
>  Labels: s3, template, templates, url, urls
> Fix For: 4.8.0
>
>
> In for example the 'volume_store_ref' table the 'download_url' field is a 
> VARCHAR(255).
> This is to short for URLs. Same goes for the 'object_datastore_ref' table and 
> the 'url' field.
> With pre-signed S3 URLs this will cause issues.
> The fix would be to change the database fields to a VARCHAR(2048). No 
> code-change is required.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (CLOUDSTACK-9238) URL fields in database are to small. Cause malformed URLs

[ASF subversion and git services (JIRA)]

[ 
https://issues.apache.org/jira/browse/CLOUDSTACK-9238?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15304613#comment-15304613
 ] 

ASF subversion and git services commented on CLOUDSTACK-9238:
-

Commit 121b3d64039ad32b97e62be8e78b09db24e3b3c9 in cloudstack's branch 
refs/heads/master from [~williamstev...@gmail.com]
[ https://git-wip-us.apache.org/repos/asf?p=cloudstack.git;h=121b3d6 ]

Merge pull request #1567 from exoscale/CLOUDSTACK-9238

CLOUDSTACK-9238: Fix URL length to 2048 for all url fields in VOI will update 
the PR to add max field length in the API commands too

* pr/1567:
  API: update url field max length
  not needed on host table
  Fix URL length to 2048 for all url fields in VO

Signed-off-by: Will Stevens 


> URL fields in database are to small. Cause malformed URLs
> -
>
> Key: CLOUDSTACK-9238
> URL: https://issues.apache.org/jira/browse/CLOUDSTACK-9238
> Project: CloudStack
>  Issue Type: Bug
>  Security Level: Public(Anyone can view this level - this is the 
> default.) 
>  Components: Management Server
>Affects Versions: 4.6.0, 4.7.0, 4.8.0
>Reporter: Wido den Hollander
>Assignee: Wido den Hollander
>  Labels: s3, template, templates, url, urls
> Fix For: 4.8.0
>
>
> In for example the 'volume_store_ref' table the 'download_url' field is a 
> VARCHAR(255).
> This is to short for URLs. Same goes for the 'object_datastore_ref' table and 
> the 'url' field.
> With pre-signed S3 URLs this will cause issues.
> The fix would be to change the database fields to a VARCHAR(2048). No 
> code-change is required.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (CLOUDSTACK-9238) URL fields in database are to small. Cause malformed URLs

[ASF subversion and git services (JIRA)]

[ 
https://issues.apache.org/jira/browse/CLOUDSTACK-9238?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15304612#comment-15304612
 ] 

ASF subversion and git services commented on CLOUDSTACK-9238:
-

Commit 121b3d64039ad32b97e62be8e78b09db24e3b3c9 in cloudstack's branch 
refs/heads/master from [~williamstev...@gmail.com]
[ https://git-wip-us.apache.org/repos/asf?p=cloudstack.git;h=121b3d6 ]

Merge pull request #1567 from exoscale/CLOUDSTACK-9238

CLOUDSTACK-9238: Fix URL length to 2048 for all url fields in VOI will update 
the PR to add max field length in the API commands too

* pr/1567:
  API: update url field max length
  not needed on host table
  Fix URL length to 2048 for all url fields in VO

Signed-off-by: Will Stevens 


> URL fields in database are to small. Cause malformed URLs
> -
>
> Key: CLOUDSTACK-9238
> URL: https://issues.apache.org/jira/browse/CLOUDSTACK-9238
> Project: CloudStack
>  Issue Type: Bug
>  Security Level: Public(Anyone can view this level - this is the 
> default.) 
>  Components: Management Server
>Affects Versions: 4.6.0, 4.7.0, 4.8.0
>Reporter: Wido den Hollander
>Assignee: Wido den Hollander
>  Labels: s3, template, templates, url, urls
> Fix For: 4.8.0
>
>
> In for example the 'volume_store_ref' table the 'download_url' field is a 
> VARCHAR(255).
> This is to short for URLs. Same goes for the 'object_datastore_ref' table and 
> the 'url' field.
> With pre-signed S3 URLs this will cause issues.
> The fix would be to change the database fields to a VARCHAR(2048). No 
> code-change is required.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (CLOUDSTACK-9238) URL fields in database are to small. Cause malformed URLs

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/CLOUDSTACK-9238?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15304536#comment-15304536
 ] 

ASF GitHub Bot commented on CLOUDSTACK-9238:


Github user swill commented on the pull request:

https://github.com/apache/cloudstack/pull/1567#issuecomment-23495
  
This is coming back clean.  I will merge it...


> URL fields in database are to small. Cause malformed URLs
> -
>
> Key: CLOUDSTACK-9238
> URL: https://issues.apache.org/jira/browse/CLOUDSTACK-9238
> Project: CloudStack
>  Issue Type: Bug
>  Security Level: Public(Anyone can view this level - this is the 
> default.) 
>  Components: Management Server
>Affects Versions: 4.6.0, 4.7.0, 4.8.0
>Reporter: Wido den Hollander
>Assignee: Wido den Hollander
>  Labels: s3, template, templates, url, urls
> Fix For: 4.8.0
>
>
> In for example the 'volume_store_ref' table the 'download_url' field is a 
> VARCHAR(255).
> This is to short for URLs. Same goes for the 'object_datastore_ref' table and 
> the 'url' field.
> With pre-signed S3 URLs this will cause issues.
> The fix would be to change the database fields to a VARCHAR(2048). No 
> code-change is required.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (CLOUDSTACK-9238) URL fields in database are to small. Cause malformed URLs

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/CLOUDSTACK-9238?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15304531#comment-15304531
 ] 

ASF GitHub Bot commented on CLOUDSTACK-9238:


Github user swill commented on the pull request:

https://github.com/apache/cloudstack/pull/1567#issuecomment-22745
  


### CI RESULTS

```
Tests Run: 85
  Skipped: 0
   Failed: 0
   Errors: 0
 Duration: 3h 56m 33s
```



**Associated Uploads**

**`/tmp/MarvinLogs/DeployDataCenter__May_27_2016_15_27_02_O0ILJL:`**
* 
[dc_entries.obj](https://objects-east.cloud.ca/v1/e465abe2f9ae4478b9fff416eab61bd9/PR1567/tmp/MarvinLogs/DeployDataCenter__May_27_2016_15_27_02_O0ILJL/dc_entries.obj)
* 
[failed_plus_exceptions.txt](https://objects-east.cloud.ca/v1/e465abe2f9ae4478b9fff416eab61bd9/PR1567/tmp/MarvinLogs/DeployDataCenter__May_27_2016_15_27_02_O0ILJL/failed_plus_exceptions.txt)
* 
[runinfo.txt](https://objects-east.cloud.ca/v1/e465abe2f9ae4478b9fff416eab61bd9/PR1567/tmp/MarvinLogs/DeployDataCenter__May_27_2016_15_27_02_O0ILJL/runinfo.txt)

**`/tmp/MarvinLogs/test_network_VEWE3L:`**
* 
[failed_plus_exceptions.txt](https://objects-east.cloud.ca/v1/e465abe2f9ae4478b9fff416eab61bd9/PR1567/tmp/MarvinLogs/test_network_VEWE3L/failed_plus_exceptions.txt)
* 
[results.txt](https://objects-east.cloud.ca/v1/e465abe2f9ae4478b9fff416eab61bd9/PR1567/tmp/MarvinLogs/test_network_VEWE3L/results.txt)
* 
[runinfo.txt](https://objects-east.cloud.ca/v1/e465abe2f9ae4478b9fff416eab61bd9/PR1567/tmp/MarvinLogs/test_network_VEWE3L/runinfo.txt)

**`/tmp/MarvinLogs/test_vpc_routers_OY7CL0:`**
* 
[failed_plus_exceptions.txt](https://objects-east.cloud.ca/v1/e465abe2f9ae4478b9fff416eab61bd9/PR1567/tmp/MarvinLogs/test_vpc_routers_OY7CL0/failed_plus_exceptions.txt)
* 
[results.txt](https://objects-east.cloud.ca/v1/e465abe2f9ae4478b9fff416eab61bd9/PR1567/tmp/MarvinLogs/test_vpc_routers_OY7CL0/results.txt)
* 
[runinfo.txt](https://objects-east.cloud.ca/v1/e465abe2f9ae4478b9fff416eab61bd9/PR1567/tmp/MarvinLogs/test_vpc_routers_OY7CL0/runinfo.txt)


Uploads will be available until `2016-07-27 02:00:00 +0200 CEST`

*Comment created by [`upr comment`](https://github.com/cloudops/upr).*



> URL fields in database are to small. Cause malformed URLs
> -
>
> Key: CLOUDSTACK-9238
> URL: https://issues.apache.org/jira/browse/CLOUDSTACK-9238
> Project: CloudStack
>  Issue Type: Bug
>  Security Level: Public(Anyone can view this level - this is the 
> default.) 
>  Components: Management Server
>Affects Versions: 4.6.0, 4.7.0, 4.8.0
>Reporter: Wido den Hollander
>Assignee: Wido den Hollander
>  Labels: s3, template, templates, url, urls
> Fix For: 4.8.0
>
>
> In for example the 'volume_store_ref' table the 'download_url' field is a 
> VARCHAR(255).
> This is to short for URLs. Same goes for the 'object_datastore_ref' table and 
> the 'url' field.
> With pre-signed S3 URLs this will cause issues.
> The fix would be to change the database fields to a VARCHAR(2048). No 
> code-change is required.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (CLOUDSTACK-9238) URL fields in database are to small. Cause malformed URLs

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/CLOUDSTACK-9238?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15304039#comment-15304039
 ] 

ASF GitHub Bot commented on CLOUDSTACK-9238:


Github user swill commented on the pull request:

https://github.com/apache/cloudstack/pull/1567#issuecomment-222141588
  
Ok, this is building in my CI environment right now, so I will see if it 
fails for me as well.  Thx...


> URL fields in database are to small. Cause malformed URLs
> -
>
> Key: CLOUDSTACK-9238
> URL: https://issues.apache.org/jira/browse/CLOUDSTACK-9238
> Project: CloudStack
>  Issue Type: Bug
>  Security Level: Public(Anyone can view this level - this is the 
> default.) 
>  Components: Management Server
>Affects Versions: 4.6.0, 4.7.0, 4.8.0
>Reporter: Wido den Hollander
>Assignee: Wido den Hollander
>  Labels: s3, template, templates, url, urls
> Fix For: 4.8.0
>
>
> In for example the 'volume_store_ref' table the 'download_url' field is a 
> VARCHAR(255).
> This is to short for URLs. Same goes for the 'object_datastore_ref' table and 
> the 'url' field.
> With pre-signed S3 URLs this will cause issues.
> The fix would be to change the database fields to a VARCHAR(2048). No 
> code-change is required.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (CLOUDSTACK-9238) URL fields in database are to small. Cause malformed URLs

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/CLOUDSTACK-9238?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15304034#comment-15304034
 ] 

ASF GitHub Bot commented on CLOUDSTACK-9238:


Github user footplus commented on the pull request:

https://github.com/apache/cloudstack/pull/1567#issuecomment-222140436
  
Seems to break the registerIso command here... Will check exactly why.

``
2016-05-27 15:01:14,211 ERROR [c.c.a.ApiServer] 
(catalina-exec-1:ctx-56423a5c ctx-9b38033e) (logid:68ac142d) unhandled 
exception executing api command: [Ljava.lang.String;@475199b7
java.lang.NullPointerException
at 
com.cloud.template.TemplateAdapterBase.prepare(TemplateAdapterBase.java:149)
at 
com.cloud.template.TemplateAdapterBase.prepare(TemplateAdapterBase.java:317)
at 
com.cloud.template.HypervisorTemplateAdapter.prepare(HypervisorTemplateAdapter.java:124)
at 
com.cloud.template.TemplateManagerImpl.registerIso(TemplateManagerImpl.java:299)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at 
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
at 
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:606)
at 
org.springframework.aop.support.AopUtils.invokeJoinpointUsingReflection(AopUtils.java:317)
at 
org.springframework.aop.framework.ReflectiveMethodInvocation.invokeJoinpoint(ReflectiveMethodInvocation.java:183)
at 
org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:150)
at 
org.apache.cloudstack.network.contrail.management.EventUtils$EventInterceptor.invoke(EventUtils.java:106)
at 
org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:161)
at 
com.cloud.event.ActionEventInterceptor.invoke(ActionEventInterceptor.java:51)
at 
org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:161)
at 
org.springframework.aop.interceptor.ExposeInvocationInterceptor.invoke(ExposeInvocationInterceptor.java:91)
at 
org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:172)
at 
org.springframework.aop.framework.JdkDynamicAopProxy.invoke(JdkDynamicAopProxy.java:204)
at com.sun.proxy.$Proxy179.registerIso(Unknown Source)
at 
org.apache.cloudstack.api.command.user.iso.RegisterIsoCmd.execute(RegisterIsoCmd.java:192)
at com.cloud.api.ApiDispatcher.dispatch(ApiDispatcher.java:150)
at com.cloud.api.ApiServer.queueCommand(ApiServer.java:698)
at com.cloud.api.ApiServer.handleRequest(ApiServer.java:529)
at com.cloud.api.ApiServlet.processRequestInContext(ApiServlet.java:297)
at com.cloud.api.ApiServlet$1.run(ApiServlet.java:127)
at 
org.apache.cloudstack.managed.context.impl.DefaultManagedContext$1.call(DefaultManagedContext.java:56)
at 
org.apache.cloudstack.managed.context.impl.DefaultManagedContext.callWithContext(DefaultManagedContext.java:103)
at 
org.apache.cloudstack.managed.context.impl.DefaultManagedContext.runWithContext(DefaultManagedContext.java:53)
at com.cloud.api.ApiServlet.processRequest(ApiServlet.java:124)
at com.cloud.api.ApiServlet.doGet(ApiServlet.java:86)
at javax.servlet.http.HttpServlet.service(HttpServlet.java:617)
at javax.servlet.http.HttpServlet.service(HttpServlet.java:723)
at 
org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:290)
at 
org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:206)
at 
org.apache.catalina.core.StandardWrapperValve.invoke(StandardWrapperValve.java:233)
at 
org.apache.catalina.core.StandardContextValve.invoke(StandardContextValve.java:191)
at 
org.apache.catalina.core.StandardHostValve.invoke(StandardHostValve.java:127)
at 
org.apache.catalina.valves.ErrorReportValve.invoke(ErrorReportValve.java:102)
at 
org.apache.catalina.valves.AccessLogValve.invoke(AccessLogValve.java:555)
at 
org.apache.catalina.core.StandardEngineValve.invoke(StandardEngineValve.java:109)
at 
org.apache.catalina.valves.RemoteIpValve.invoke(RemoteIpValve.java:600)
at 
org.apache.catalina.connector.CoyoteAdapter.service(CoyoteAdapter.java:299)
at 
org.apache.coyote.http11.Http11NioProcessor.process(Http11NioProcessor.java:889)
at 
org.apache.coyote.http11.Http11NioProtocol$Http11ConnectionHandler.process(Http11NioProtocol.java:732)
at 
org.apache.tomcat.util.net.NioEndpoint$SocketProcessor.run(NioEndpoint.java:2274)
at 

[jira] [Commented] (CLOUDSTACK-9238) URL fields in database are to small. Cause malformed URLs

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/CLOUDSTACK-9238?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15303841#comment-15303841
 ] 

ASF GitHub Bot commented on CLOUDSTACK-9238:


Github user wido closed the pull request at:

https://github.com/apache/cloudstack/pull/1562


> URL fields in database are to small. Cause malformed URLs
> -
>
> Key: CLOUDSTACK-9238
> URL: https://issues.apache.org/jira/browse/CLOUDSTACK-9238
> Project: CloudStack
>  Issue Type: Bug
>  Security Level: Public(Anyone can view this level - this is the 
> default.) 
>  Components: Management Server
>Affects Versions: 4.6.0, 4.7.0, 4.8.0
>Reporter: Wido den Hollander
>Assignee: Wido den Hollander
>  Labels: s3, template, templates, url, urls
> Fix For: 4.8.0
>
>
> In for example the 'volume_store_ref' table the 'download_url' field is a 
> VARCHAR(255).
> This is to short for URLs. Same goes for the 'object_datastore_ref' table and 
> the 'url' field.
> With pre-signed S3 URLs this will cause issues.
> The fix would be to change the database fields to a VARCHAR(2048). No 
> code-change is required.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (CLOUDSTACK-9238) URL fields in database are to small. Cause malformed URLs

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/CLOUDSTACK-9238?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15303800#comment-15303800
 ] 

ASF GitHub Bot commented on CLOUDSTACK-9238:


Github user karuturi commented on the pull request:

https://github.com/apache/cloudstack/pull/1567#issuecomment-222099870
  
ahh.. my bad.. thanks for the clarification.
so, we need to introduce a new CommandType.URL for api parameters and use 
java.net.URL in VOs.

LGTM in the current state as well as this fixes the issue. 



> URL fields in database are to small. Cause malformed URLs
> -
>
> Key: CLOUDSTACK-9238
> URL: https://issues.apache.org/jira/browse/CLOUDSTACK-9238
> Project: CloudStack
>  Issue Type: Bug
>  Security Level: Public(Anyone can view this level - this is the 
> default.) 
>  Components: Management Server
>Affects Versions: 4.6.0, 4.7.0, 4.8.0
>Reporter: Wido den Hollander
>Assignee: Wido den Hollander
>  Labels: s3, template, templates, url, urls
> Fix For: 4.8.0
>
>
> In for example the 'volume_store_ref' table the 'download_url' field is a 
> VARCHAR(255).
> This is to short for URLs. Same goes for the 'object_datastore_ref' table and 
> the 'url' field.
> With pre-signed S3 URLs this will cause issues.
> The fix would be to change the database fields to a VARCHAR(2048). No 
> code-change is required.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (CLOUDSTACK-9238) URL fields in database are to small. Cause malformed URLs

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/CLOUDSTACK-9238?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15303774#comment-15303774
 ] 

ASF GitHub Bot commented on CLOUDSTACK-9238:


Github user marcaurele commented on the pull request:

https://github.com/apache/cloudstack/pull/1567#issuecomment-222094671
  
Because not all varchar columns have been updated to a length of 2048 in 
the DB, so there will be a mismatche between API, VO and the database.


> URL fields in database are to small. Cause malformed URLs
> -
>
> Key: CLOUDSTACK-9238
> URL: https://issues.apache.org/jira/browse/CLOUDSTACK-9238
> Project: CloudStack
>  Issue Type: Bug
>  Security Level: Public(Anyone can view this level - this is the 
> default.) 
>  Components: Management Server
>Affects Versions: 4.6.0, 4.7.0, 4.8.0
>Reporter: Wido den Hollander
>Assignee: Wido den Hollander
>  Labels: s3, template, templates, url, urls
> Fix For: 4.8.0
>
>
> In for example the 'volume_store_ref' table the 'download_url' field is a 
> VARCHAR(255).
> This is to short for URLs. Same goes for the 'object_datastore_ref' table and 
> the 'url' field.
> With pre-signed S3 URLs this will cause issues.
> The fix would be to change the database fields to a VARCHAR(2048). No 
> code-change is required.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (CLOUDSTACK-9238) URL fields in database are to small. Cause malformed URLs

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/CLOUDSTACK-9238?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15303767#comment-15303767
 ] 

ASF GitHub Bot commented on CLOUDSTACK-9238:


Github user karuturi commented on a diff in the pull request:

https://github.com/apache/cloudstack/pull/1567#discussion_r64875174
  
--- Diff: setup/db/db/schema-471to480.sql ---
@@ -27,3 +27,4 @@ ALTER TABLE `cloud`.`image_store` MODIFY COLUMN `url` 
varchar(2048);
 ALTER TABLE `cloud`.`template_store_ref` MODIFY COLUMN `url` varchar(2048);
 ALTER TABLE `cloud`.`volume_store_ref` MODIFY COLUMN `url` varchar(2048);
 ALTER TABLE `cloud`.`volume_store_ref` MODIFY COLUMN `download_url` 
varchar(2048);
+ALTER TABLE `cloud`.`upload` MODIFY COLUMN `url` varchar(2048);
--- End diff --

I think this table isnt used anymore. 


> URL fields in database are to small. Cause malformed URLs
> -
>
> Key: CLOUDSTACK-9238
> URL: https://issues.apache.org/jira/browse/CLOUDSTACK-9238
> Project: CloudStack
>  Issue Type: Bug
>  Security Level: Public(Anyone can view this level - this is the 
> default.) 
>  Components: Management Server
>Affects Versions: 4.6.0, 4.7.0, 4.8.0
>Reporter: Wido den Hollander
>Assignee: Wido den Hollander
>  Labels: s3, template, templates, url, urls
> Fix For: 4.8.0
>
>
> In for example the 'volume_store_ref' table the 'download_url' field is a 
> VARCHAR(255).
> This is to short for URLs. Same goes for the 'object_datastore_ref' table and 
> the 'url' field.
> With pre-signed S3 URLs this will cause issues.
> The fix would be to change the database fields to a VARCHAR(2048). No 
> code-change is required.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (CLOUDSTACK-9238) URL fields in database are to small. Cause malformed URLs

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/CLOUDSTACK-9238?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15303763#comment-15303763
 ] 

ASF GitHub Bot commented on CLOUDSTACK-9238:


Github user karuturi commented on the pull request:

https://github.com/apache/cloudstack/pull/1567#issuecomment-222093757
  
Why dont we change the default length in 
[GenericDaoBase](https://github.com/apache/cloudstack/blob/master/framework/db/src/com/cloud/utils/db/GenericDaoBase.java#L1510)
 and 
[api.Parameter](https://github.com/apache/cloudstack/blob/master/api/src/org/apache/cloudstack/api/Parameter.java#L47)
 ?
If possible, get it from some global constant


> URL fields in database are to small. Cause malformed URLs
> -
>
> Key: CLOUDSTACK-9238
> URL: https://issues.apache.org/jira/browse/CLOUDSTACK-9238
> Project: CloudStack
>  Issue Type: Bug
>  Security Level: Public(Anyone can view this level - this is the 
> default.) 
>  Components: Management Server
>Affects Versions: 4.6.0, 4.7.0, 4.8.0
>Reporter: Wido den Hollander
>Assignee: Wido den Hollander
>  Labels: s3, template, templates, url, urls
> Fix For: 4.8.0
>
>
> In for example the 'volume_store_ref' table the 'download_url' field is a 
> VARCHAR(255).
> This is to short for URLs. Same goes for the 'object_datastore_ref' table and 
> the 'url' field.
> With pre-signed S3 URLs this will cause issues.
> The fix would be to change the database fields to a VARCHAR(2048). No 
> code-change is required.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (CLOUDSTACK-9238) URL fields in database are to small. Cause malformed URLs

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/CLOUDSTACK-9238?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15303632#comment-15303632
 ] 

ASF GitHub Bot commented on CLOUDSTACK-9238:


Github user marcaurele commented on the pull request:

https://github.com/apache/cloudstack/pull/1341#issuecomment-222071694
  
I guess a PR is easier, so here you go 
https://github.com/apache/cloudstack/pull/1567


> URL fields in database are to small. Cause malformed URLs
> -
>
> Key: CLOUDSTACK-9238
> URL: https://issues.apache.org/jira/browse/CLOUDSTACK-9238
> Project: CloudStack
>  Issue Type: Bug
>  Security Level: Public(Anyone can view this level - this is the 
> default.) 
>  Components: Management Server
>Affects Versions: 4.6.0, 4.7.0, 4.8.0
>Reporter: Wido den Hollander
>Assignee: Wido den Hollander
>  Labels: s3, template, templates, url, urls
> Fix For: 4.8.0
>
>
> In for example the 'volume_store_ref' table the 'download_url' field is a 
> VARCHAR(255).
> This is to short for URLs. Same goes for the 'object_datastore_ref' table and 
> the 'url' field.
> With pre-signed S3 URLs this will cause issues.
> The fix would be to change the database fields to a VARCHAR(2048). No 
> code-change is required.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (CLOUDSTACK-9238) URL fields in database are to small. Cause malformed URLs

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/CLOUDSTACK-9238?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15303609#comment-15303609
 ] 

ASF GitHub Bot commented on CLOUDSTACK-9238:


GitHub user marcaurele opened a pull request:

https://github.com/apache/cloudstack/pull/1567

CLOUDSTACK-9238: Fix URL length to 2048 for all url fields in VO

I will update the PR to add max field length in the API commands too

You can merge this pull request into a Git repository by running:

$ git pull https://github.com/exoscale/cloudstack CLOUDSTACK-9238

Alternatively you can review and apply these changes as the patch at:

https://github.com/apache/cloudstack/pull/1567.patch

To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:

This closes #1567


commit a59ee03fd79806dcb3a8842c318e2cd06895183f
Author: Marc-Aurèle Brothier 
Date:   2016-05-27T06:16:05Z

Fix URL length to 2048 for all url fields in VO




> URL fields in database are to small. Cause malformed URLs
> -
>
> Key: CLOUDSTACK-9238
> URL: https://issues.apache.org/jira/browse/CLOUDSTACK-9238
> Project: CloudStack
>  Issue Type: Bug
>  Security Level: Public(Anyone can view this level - this is the 
> default.) 
>  Components: Management Server
>Affects Versions: 4.6.0, 4.7.0, 4.8.0
>Reporter: Wido den Hollander
>Assignee: Wido den Hollander
>  Labels: s3, template, templates, url, urls
> Fix For: 4.8.0
>
>
> In for example the 'volume_store_ref' table the 'download_url' field is a 
> VARCHAR(255).
> This is to short for URLs. Same goes for the 'object_datastore_ref' table and 
> the 'url' field.
> With pre-signed S3 URLs this will cause issues.
> The fix would be to change the database fields to a VARCHAR(2048). No 
> code-change is required.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (CLOUDSTACK-9238) URL fields in database are to small. Cause malformed URLs

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/CLOUDSTACK-9238?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15303582#comment-15303582
 ] 

ASF GitHub Bot commented on CLOUDSTACK-9238:


Github user marcaurele commented on the pull request:

https://github.com/apache/cloudstack/pull/1341#issuecomment-222067277
  
Hi! Talking with @footplus on the mailing list, I realized that this patch 
is not correct. It's missing modifications on the VO objects to let the String 
holds more than 255 characters because in the `GenericDaoBase`, `String` fields 
are by default of 255 chars, see:

https://github.com/apache/cloudstack/blob/master/framework/db/src/com/cloud/utils/db/GenericDaoBase.java#L1510

This PR should be updated, or should there be a new one to add those 
changes?


> URL fields in database are to small. Cause malformed URLs
> -
>
> Key: CLOUDSTACK-9238
> URL: https://issues.apache.org/jira/browse/CLOUDSTACK-9238
> Project: CloudStack
>  Issue Type: Bug
>  Security Level: Public(Anyone can view this level - this is the 
> default.) 
>  Components: Management Server
>Affects Versions: 4.6.0, 4.7.0, 4.8.0
>Reporter: Wido den Hollander
>Assignee: Wido den Hollander
>  Labels: s3, template, templates, url, urls
> Fix For: 4.8.0
>
>
> In for example the 'volume_store_ref' table the 'download_url' field is a 
> VARCHAR(255).
> This is to short for URLs. Same goes for the 'object_datastore_ref' table and 
> the 'url' field.
> With pre-signed S3 URLs this will cause issues.
> The fix would be to change the database fields to a VARCHAR(2048). No 
> code-change is required.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (CLOUDSTACK-9238) URL fields in database are to small. Cause malformed URLs

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/CLOUDSTACK-9238?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15302249#comment-15302249
 ] 

ASF GitHub Bot commented on CLOUDSTACK-9238:


Github user swill commented on the pull request:

https://github.com/apache/cloudstack/pull/1562#issuecomment-221904814
  


### CI RESULTS

```
Tests Run: 82
  Skipped: 0
   Failed: 0
   Errors: 3
 Duration: 8h 17m 54s
```

**Summary of the problem(s):**
```
ERROR: test suite for 
--
Traceback (most recent call last):
  File 
"/usr/lib/python2.7/site-packages/nose-1.3.7-py2.7.egg/nose/suite.py", line 
209, in run
self.setUp()
  File 
"/usr/lib/python2.7/site-packages/nose-1.3.7-py2.7.egg/nose/suite.py", line 
292, in setUp
self.setupContext(ancestor)
  File 
"/usr/lib/python2.7/site-packages/nose-1.3.7-py2.7.egg/nose/suite.py", line 
315, in setupContext
try_run(context, names)
  File 
"/usr/lib/python2.7/site-packages/nose-1.3.7-py2.7.egg/nose/util.py", line 471, 
in try_run
return func()
  File "/data/git/cs1/cloudstack/test/integration/smoke/test_vpc_vpn.py", 
line 835, in setUpClass
cls.template.download(cls.apiclient)
  File "/usr/lib/python2.7/site-packages/marvin/lib/base.py", line 1350, in 
download
elif 'Downloaded' in template.status:
TypeError: argument of type 'NoneType' is not iterable
--
Additional details in: /tmp/MarvinLogs/test_network_DL503B/results.txt
```

```
ERROR: test suite for 
--
Traceback (most recent call last):
  File 
"/usr/lib/python2.7/site-packages/nose-1.3.7-py2.7.egg/nose/suite.py", line 
209, in run
self.setUp()
  File 
"/usr/lib/python2.7/site-packages/nose-1.3.7-py2.7.egg/nose/suite.py", line 
292, in setUp
self.setupContext(ancestor)
  File 
"/usr/lib/python2.7/site-packages/nose-1.3.7-py2.7.egg/nose/suite.py", line 
315, in setupContext
try_run(context, names)
  File 
"/usr/lib/python2.7/site-packages/nose-1.3.7-py2.7.egg/nose/util.py", line 471, 
in try_run
return func()
  File "/data/git/cs1/cloudstack/test/integration/smoke/test_vpc_vpn.py", 
line 293, in setUpClass
cls.template.download(cls.apiclient)
  File "/usr/lib/python2.7/site-packages/marvin/lib/base.py", line 1350, in 
download
elif 'Downloaded' in template.status:
TypeError: argument of type 'NoneType' is not iterable
--
Additional details in: /tmp/MarvinLogs/test_network_DL503B/results.txt
```

```
ERROR: test suite for 
--
Traceback (most recent call last):
  File 
"/usr/lib/python2.7/site-packages/nose-1.3.7-py2.7.egg/nose/suite.py", line 
209, in run
self.setUp()
  File 
"/usr/lib/python2.7/site-packages/nose-1.3.7-py2.7.egg/nose/suite.py", line 
292, in setUp
self.setupContext(ancestor)
  File 
"/usr/lib/python2.7/site-packages/nose-1.3.7-py2.7.egg/nose/suite.py", line 
315, in setupContext
try_run(context, names)
  File 
"/usr/lib/python2.7/site-packages/nose-1.3.7-py2.7.egg/nose/util.py", line 471, 
in try_run
return func()
  File "/data/git/cs1/cloudstack/test/integration/smoke/test_vpc_vpn.py", 
line 472, in setUpClass
cls.template.download(cls.apiclient)
  File "/usr/lib/python2.7/site-packages/marvin/lib/base.py", line 1350, in 
download
elif 'Downloaded' in template.status:
TypeError: argument of type 'NoneType' is not iterable
--
Additional details in: /tmp/MarvinLogs/test_network_DL503B/results.txt
```



**Associated Uploads**

**`/tmp/MarvinLogs/DeployDataCenter__May_26_2016_04_59_06_DR371G:`**
* 
[dc_entries.obj](https://objects-east.cloud.ca/v1/e465abe2f9ae4478b9fff416eab61bd9/PR1562/tmp/MarvinLogs/DeployDataCenter__May_26_2016_04_59_06_DR371G/dc_entries.obj)
* 
[failed_plus_exceptions.txt](https://objects-east.cloud.ca/v1/e465abe2f9ae4478b9fff416eab61bd9/PR1562/tmp/MarvinLogs/DeployDataCenter__May_26_2016_04_59_06_DR371G/failed_plus_exceptions.txt)
* 
[runinfo.txt](https://objects-east.cloud.ca/v1/e465abe2f9ae4478b9fff416eab61bd9/PR1562/tmp/MarvinLogs/DeployDataCenter__May_26_2016_04_59_06_DR371G/runinfo.txt)

**`/tmp/MarvinLogs/test_network_DL503B:`**
* 
[failed_plus_exceptions.txt](https://objects-east.cloud.ca/v1/e465abe2f9ae4478b9fff416eab61bd9/PR1562/tmp/MarvinLogs/test_network_DL503B/failed_plus_exceptions.txt)
  

[jira] [Commented] (CLOUDSTACK-9238) URL fields in database are to small. Cause malformed URLs

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/CLOUDSTACK-9238?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15302174#comment-15302174
 ] 

ASF GitHub Bot commented on CLOUDSTACK-9238:


Github user footplus commented on the pull request:

https://github.com/apache/cloudstack/pull/1562#issuecomment-221893016
  
Maybe you meant to change on this table:

``
alter table `vm_template` modify column `url` varchar(2048) DEFAULT NULL 
COMMENT 'the url where the template exists externally';
``


> URL fields in database are to small. Cause malformed URLs
> -
>
> Key: CLOUDSTACK-9238
> URL: https://issues.apache.org/jira/browse/CLOUDSTACK-9238
> Project: CloudStack
>  Issue Type: Bug
>  Security Level: Public(Anyone can view this level - this is the 
> default.) 
>  Components: Management Server
>Affects Versions: 4.6.0, 4.7.0, 4.8.0
>Reporter: Wido den Hollander
>Assignee: Wido den Hollander
>  Labels: s3, template, templates, url, urls
> Fix For: 4.8.0
>
>
> In for example the 'volume_store_ref' table the 'download_url' field is a 
> VARCHAR(255).
> This is to short for URLs. Same goes for the 'object_datastore_ref' table and 
> the 'url' field.
> With pre-signed S3 URLs this will cause issues.
> The fix would be to change the database fields to a VARCHAR(2048). No 
> code-change is required.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (CLOUDSTACK-9238) URL fields in database are to small. Cause malformed URLs

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/CLOUDSTACK-9238?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15302085#comment-15302085
 ] 

ASF GitHub Bot commented on CLOUDSTACK-9238:


Github user footplus commented on the pull request:

https://github.com/apache/cloudstack/pull/1562#issuecomment-221876586
  
@wido I have the 255-chars problems here too, and this particular table is 
empty; did you add a patch on the API too to unlock the API limitation to 255 
chars ?


> URL fields in database are to small. Cause malformed URLs
> -
>
> Key: CLOUDSTACK-9238
> URL: https://issues.apache.org/jira/browse/CLOUDSTACK-9238
> Project: CloudStack
>  Issue Type: Bug
>  Security Level: Public(Anyone can view this level - this is the 
> default.) 
>  Components: Management Server
>Affects Versions: 4.6.0, 4.7.0, 4.8.0
>Reporter: Wido den Hollander
>Assignee: Wido den Hollander
>  Labels: s3, template, templates, url, urls
> Fix For: 4.8.0
>
>
> In for example the 'volume_store_ref' table the 'download_url' field is a 
> VARCHAR(255).
> This is to short for URLs. Same goes for the 'object_datastore_ref' table and 
> the 'url' field.
> With pre-signed S3 URLs this will cause issues.
> The fix would be to change the database fields to a VARCHAR(2048). No 
> code-change is required.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (CLOUDSTACK-9238) URL fields in database are to small. Cause malformed URLs

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/CLOUDSTACK-9238?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15301883#comment-15301883
 ] 

ASF GitHub Bot commented on CLOUDSTACK-9238:


Github user karuturi commented on the pull request:

https://github.com/apache/cloudstack/pull/1562#issuecomment-221825423
  
commit 5c35bd1b3e0184842ed80a9c465549c2a61f5fd9 copied all the data from 
template_host_ref to template_store_ref 



> URL fields in database are to small. Cause malformed URLs
> -
>
> Key: CLOUDSTACK-9238
> URL: https://issues.apache.org/jira/browse/CLOUDSTACK-9238
> Project: CloudStack
>  Issue Type: Bug
>  Security Level: Public(Anyone can view this level - this is the 
> default.) 
>  Components: Management Server
>Affects Versions: 4.6.0, 4.7.0, 4.8.0
>Reporter: Wido den Hollander
>Assignee: Wido den Hollander
>  Labels: s3, template, templates, url, urls
> Fix For: 4.8.0
>
>
> In for example the 'volume_store_ref' table the 'download_url' field is a 
> VARCHAR(255).
> This is to short for URLs. Same goes for the 'object_datastore_ref' table and 
> the 'url' field.
> With pre-signed S3 URLs this will cause issues.
> The fix would be to change the database fields to a VARCHAR(2048). No 
> code-change is required.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (CLOUDSTACK-9238) URL fields in database are to small. Cause malformed URLs

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/CLOUDSTACK-9238?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15301879#comment-15301879
 ] 

ASF GitHub Bot commented on CLOUDSTACK-9238:


Github user wido commented on the pull request:

https://github.com/apache/cloudstack/pull/1562#issuecomment-221823209
  
@karuturi Are you sure about that?


> URL fields in database are to small. Cause malformed URLs
> -
>
> Key: CLOUDSTACK-9238
> URL: https://issues.apache.org/jira/browse/CLOUDSTACK-9238
> Project: CloudStack
>  Issue Type: Bug
>  Security Level: Public(Anyone can view this level - this is the 
> default.) 
>  Components: Management Server
>Affects Versions: 4.6.0, 4.7.0, 4.8.0
>Reporter: Wido den Hollander
>Assignee: Wido den Hollander
>  Labels: s3, template, templates, url, urls
> Fix For: 4.8.0
>
>
> In for example the 'volume_store_ref' table the 'download_url' field is a 
> VARCHAR(255).
> This is to short for URLs. Same goes for the 'object_datastore_ref' table and 
> the 'url' field.
> With pre-signed S3 URLs this will cause issues.
> The fix would be to change the database fields to a VARCHAR(2048). No 
> code-change is required.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (CLOUDSTACK-9238) URL fields in database are to small. Cause malformed URLs

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/CLOUDSTACK-9238?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15301872#comment-15301872
 ] 

ASF GitHub Bot commented on CLOUDSTACK-9238:


Github user karuturi commented on the pull request:

https://github.com/apache/cloudstack/pull/1562#issuecomment-221821581
  
This table is not used anymore. the table should be dropped.


> URL fields in database are to small. Cause malformed URLs
> -
>
> Key: CLOUDSTACK-9238
> URL: https://issues.apache.org/jira/browse/CLOUDSTACK-9238
> Project: CloudStack
>  Issue Type: Bug
>  Security Level: Public(Anyone can view this level - this is the 
> default.) 
>  Components: Management Server
>Affects Versions: 4.6.0, 4.7.0, 4.8.0
>Reporter: Wido den Hollander
>Assignee: Wido den Hollander
>  Labels: s3, template, templates, url, urls
> Fix For: 4.8.0
>
>
> In for example the 'volume_store_ref' table the 'download_url' field is a 
> VARCHAR(255).
> This is to short for URLs. Same goes for the 'object_datastore_ref' table and 
> the 'url' field.
> With pre-signed S3 URLs this will cause issues.
> The fix would be to change the database fields to a VARCHAR(2048). No 
> code-change is required.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (CLOUDSTACK-9238) URL fields in database are to small. Cause malformed URLs

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/CLOUDSTACK-9238?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15299859#comment-15299859
 ] 

ASF GitHub Bot commented on CLOUDSTACK-9238:


Github user rhtyd commented on the pull request:

https://github.com/apache/cloudstack/pull/1562#issuecomment-221538713
  
LGTM, simple fix.
Tested on local database.

tag:easypr


> URL fields in database are to small. Cause malformed URLs
> -
>
> Key: CLOUDSTACK-9238
> URL: https://issues.apache.org/jira/browse/CLOUDSTACK-9238
> Project: CloudStack
>  Issue Type: Bug
>  Security Level: Public(Anyone can view this level - this is the 
> default.) 
>  Components: Management Server
>Affects Versions: 4.6.0, 4.7.0, 4.8.0
>Reporter: Wido den Hollander
>Assignee: Wido den Hollander
>  Labels: s3, template, templates, url, urls
> Fix For: 4.8.0
>
>
> In for example the 'volume_store_ref' table the 'download_url' field is a 
> VARCHAR(255).
> This is to short for URLs. Same goes for the 'object_datastore_ref' table and 
> the 'url' field.
> With pre-signed S3 URLs this will cause issues.
> The fix would be to change the database fields to a VARCHAR(2048). No 
> code-change is required.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (CLOUDSTACK-9238) URL fields in database are to small. Cause malformed URLs

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/CLOUDSTACK-9238?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15299855#comment-15299855
 ] 

ASF GitHub Bot commented on CLOUDSTACK-9238:


Github user wido commented on the pull request:

https://github.com/apache/cloudstack/pull/1562#issuecomment-221538332
  
@rhtyd: Could you take a look? As you can see, it's a very small fix.


> URL fields in database are to small. Cause malformed URLs
> -
>
> Key: CLOUDSTACK-9238
> URL: https://issues.apache.org/jira/browse/CLOUDSTACK-9238
> Project: CloudStack
>  Issue Type: Bug
>  Security Level: Public(Anyone can view this level - this is the 
> default.) 
>  Components: Management Server
>Affects Versions: 4.6.0, 4.7.0, 4.8.0
>Reporter: Wido den Hollander
>Assignee: Wido den Hollander
>  Labels: s3, template, templates, url, urls
> Fix For: 4.8.0
>
>
> In for example the 'volume_store_ref' table the 'download_url' field is a 
> VARCHAR(255).
> This is to short for URLs. Same goes for the 'object_datastore_ref' table and 
> the 'url' field.
> With pre-signed S3 URLs this will cause issues.
> The fix would be to change the database fields to a VARCHAR(2048). No 
> code-change is required.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (CLOUDSTACK-9238) URL fields in database are to small. Cause malformed URLs

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/CLOUDSTACK-9238?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15299808#comment-15299808
 ] 

ASF GitHub Bot commented on CLOUDSTACK-9238:


Github user wido commented on the pull request:

https://github.com/apache/cloudstack/pull/1562#issuecomment-221531859
  
@swill: Still time to get this one in?

This fix is running in production on our clouds. It is just a ALTER TABLE 
to increase the size of a database field.


> URL fields in database are to small. Cause malformed URLs
> -
>
> Key: CLOUDSTACK-9238
> URL: https://issues.apache.org/jira/browse/CLOUDSTACK-9238
> Project: CloudStack
>  Issue Type: Bug
>  Security Level: Public(Anyone can view this level - this is the 
> default.) 
>  Components: Management Server
>Affects Versions: 4.6.0, 4.7.0, 4.8.0
>Reporter: Wido den Hollander
>Assignee: Wido den Hollander
>  Labels: s3, template, templates, url, urls
> Fix For: 4.8.0
>
>
> In for example the 'volume_store_ref' table the 'download_url' field is a 
> VARCHAR(255).
> This is to short for URLs. Same goes for the 'object_datastore_ref' table and 
> the 'url' field.
> With pre-signed S3 URLs this will cause issues.
> The fix would be to change the database fields to a VARCHAR(2048). No 
> code-change is required.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (CLOUDSTACK-9238) URL fields in database are to small. Cause malformed URLs

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/CLOUDSTACK-9238?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15299805#comment-15299805
 ] 

ASF GitHub Bot commented on CLOUDSTACK-9238:


GitHub user wido opened a pull request:

https://github.com/apache/cloudstack/pull/1562

CLOUDSTACK-9238: Change template_host_ref.url to varchar(2048)

In Pull Request #1341 / commit a171bb this was fixed for many other
URL fields, but this was not corrected and is still a varchar(255)

Change this field to a varchar(2048) to accomodate longer URLs in
the database.

You can merge this pull request into a Git repository by running:

$ git pull https://github.com/wido/cloudstack CLOUDSTACK-9238

Alternatively you can review and apply these changes as the patch at:

https://github.com/apache/cloudstack/pull/1562.patch

To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:

This closes #1562


commit a77a301f715ea64f3c5e0ff8b37923663868f1a6
Author: Wido den Hollander 
Date:   2016-05-25T10:12:58Z

CLOUDSTACK-9238: Change template_host_ref.url to varchar(2048)

In Pull Request #1341 / commit a171bb this was fixed for many other
URL fields, but this was not corrected and is still a varchar(255)

Change this field to a varchar(2048) to accomodate longer URLs in
the database.




> URL fields in database are to small. Cause malformed URLs
> -
>
> Key: CLOUDSTACK-9238
> URL: https://issues.apache.org/jira/browse/CLOUDSTACK-9238
> Project: CloudStack
>  Issue Type: Bug
>  Security Level: Public(Anyone can view this level - this is the 
> default.) 
>  Components: Management Server
>Affects Versions: 4.6.0, 4.7.0, 4.8.0
>Reporter: Wido den Hollander
>Assignee: Wido den Hollander
>  Labels: s3, template, templates, url, urls
> Fix For: 4.8.0
>
>
> In for example the 'volume_store_ref' table the 'download_url' field is a 
> VARCHAR(255).
> This is to short for URLs. Same goes for the 'object_datastore_ref' table and 
> the 'url' field.
> With pre-signed S3 URLs this will cause issues.
> The fix would be to change the database fields to a VARCHAR(2048). No 
> code-change is required.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (CLOUDSTACK-9238) URL fields in database are to small. Cause malformed URLs

[ASF subversion and git services (JIRA)]

[ 
https://issues.apache.org/jira/browse/CLOUDSTACK-9238?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15103312#comment-15103312
 ] 

ASF subversion and git services commented on CLOUDSTACK-9238:
-

Commit a171bbc96dc4c44734d6c14265347f9db791a0cb in cloudstack's branch 
refs/heads/master from [~widodh]
[ https://git-wip-us.apache.org/repos/asf?p=cloudstack.git;h=a171bbc ]

CLOUDSTACK-9238: Increase URL fields to 2048 charachters from 255

255 characters is to small for various URLs like S3 pre-signed URLs.

This causes one or more characters to be chopped of the end of the URL
and this renders them useless.

Internally in the code all URLs are passed as Strings and they are not
sized limited. This was purely in the database.

Other URL fields in the database were already 2048 characters.

This limit was introduced in the 4.1 to 4.2 upgrade when Object storage
like S3 and Swift was introduced in CloudStack for Secondary Storage.


> URL fields in database are to small. Cause malformed URLs
> -
>
> Key: CLOUDSTACK-9238
> URL: https://issues.apache.org/jira/browse/CLOUDSTACK-9238
> Project: CloudStack
>  Issue Type: Bug
>  Security Level: Public(Anyone can view this level - this is the 
> default.) 
>  Components: Management Server
>Affects Versions: 4.6.0, 4.7.0, 4.8.0
>Reporter: Wido den Hollander
>Assignee: Wido den Hollander
>  Labels: s3, template, templates, url, urls
> Fix For: 4.8.0
>
>
> In for example the 'volume_store_ref' table the 'download_url' field is a 
> VARCHAR(255).
> This is to short for URLs. Same goes for the 'object_datastore_ref' table and 
> the 'url' field.
> With pre-signed S3 URLs this will cause issues.
> The fix would be to change the database fields to a VARCHAR(2048). No 
> code-change is required.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (CLOUDSTACK-9238) URL fields in database are to small. Cause malformed URLs

[ASF subversion and git services (JIRA)]

[ 
https://issues.apache.org/jira/browse/CLOUDSTACK-9238?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15103314#comment-15103314
 ] 

ASF subversion and git services commented on CLOUDSTACK-9238:
-

Commit f79b3e1c2ad747d2aea6530c6672737915af13ab in cloudstack's branch 
refs/heads/master from [~remibergsma]
[ https://git-wip-us.apache.org/repos/asf?p=cloudstack.git;h=f79b3e1 ]

Merge pull request #1341 from wido/CLOUDSTACK-9238

CLOUDSTACK-9238: Increase URL fields to 2048 charachters from 255255 characters 
is to small for various URLs like S3 pre-signed URLs.

This causes one or more characters to be chopped of the end of the URL
and this renders them useless.

Internally in the code all URLs are passed as Strings and they are not
sized limited. This was purely in the database.

Other URL fields in the database were already 2048 characters.

This limit was introduced in the 4.1 to 4.2 upgrade when Object storage
like S3 and Swift was introduced in CloudStack for Secondary Storage.

* pr/1341:
  CLOUDSTACK-9238: Increase URL fields to 2048 charachters from 255

Signed-off-by: Remi Bergsma 


> URL fields in database are to small. Cause malformed URLs
> -
>
> Key: CLOUDSTACK-9238
> URL: https://issues.apache.org/jira/browse/CLOUDSTACK-9238
> Project: CloudStack
>  Issue Type: Bug
>  Security Level: Public(Anyone can view this level - this is the 
> default.) 
>  Components: Management Server
>Affects Versions: 4.6.0, 4.7.0, 4.8.0
>Reporter: Wido den Hollander
>Assignee: Wido den Hollander
>  Labels: s3, template, templates, url, urls
> Fix For: 4.8.0
>
>
> In for example the 'volume_store_ref' table the 'download_url' field is a 
> VARCHAR(255).
> This is to short for URLs. Same goes for the 'object_datastore_ref' table and 
> the 'url' field.
> With pre-signed S3 URLs this will cause issues.
> The fix would be to change the database fields to a VARCHAR(2048). No 
> code-change is required.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (CLOUDSTACK-9238) URL fields in database are to small. Cause malformed URLs

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/CLOUDSTACK-9238?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15103311#comment-15103311
 ] 

ASF GitHub Bot commented on CLOUDSTACK-9238:


Github user remibergsma commented on the pull request:

https://github.com/apache/cloudstack/pull/1341#issuecomment-172244406
  
This page has some interesting background:

http://stackoverflow.com/questions/417142/what-is-the-maximum-length-of-a-url-in-different-browsers

It seems URLs can be much longer. The question is whether we need those in 
CloudStack. For now I'm OK with 2K instead of 256. We can always bump them 
later when needed.

LGTM


> URL fields in database are to small. Cause malformed URLs
> -
>
> Key: CLOUDSTACK-9238
> URL: https://issues.apache.org/jira/browse/CLOUDSTACK-9238
> Project: CloudStack
>  Issue Type: Bug
>  Security Level: Public(Anyone can view this level - this is the 
> default.) 
>  Components: Management Server
>Affects Versions: 4.6.0, 4.7.0, 4.8.0
>Reporter: Wido den Hollander
>Assignee: Wido den Hollander
>  Labels: s3, template, templates, url, urls
> Fix For: 4.8.0
>
>
> In for example the 'volume_store_ref' table the 'download_url' field is a 
> VARCHAR(255).
> This is to short for URLs. Same goes for the 'object_datastore_ref' table and 
> the 'url' field.
> With pre-signed S3 URLs this will cause issues.
> The fix would be to change the database fields to a VARCHAR(2048). No 
> code-change is required.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (CLOUDSTACK-9238) URL fields in database are to small. Cause malformed URLs

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/CLOUDSTACK-9238?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15103316#comment-15103316
 ] 

ASF GitHub Bot commented on CLOUDSTACK-9238:


Github user asfgit closed the pull request at:

https://github.com/apache/cloudstack/pull/1341


> URL fields in database are to small. Cause malformed URLs
> -
>
> Key: CLOUDSTACK-9238
> URL: https://issues.apache.org/jira/browse/CLOUDSTACK-9238
> Project: CloudStack
>  Issue Type: Bug
>  Security Level: Public(Anyone can view this level - this is the 
> default.) 
>  Components: Management Server
>Affects Versions: 4.6.0, 4.7.0, 4.8.0
>Reporter: Wido den Hollander
>Assignee: Wido den Hollander
>  Labels: s3, template, templates, url, urls
> Fix For: 4.8.0
>
>
> In for example the 'volume_store_ref' table the 'download_url' field is a 
> VARCHAR(255).
> This is to short for URLs. Same goes for the 'object_datastore_ref' table and 
> the 'url' field.
> With pre-signed S3 URLs this will cause issues.
> The fix would be to change the database fields to a VARCHAR(2048). No 
> code-change is required.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (CLOUDSTACK-9238) URL fields in database are to small. Cause malformed URLs

[ASF subversion and git services (JIRA)]

[ 
https://issues.apache.org/jira/browse/CLOUDSTACK-9238?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15103313#comment-15103313
 ] 

ASF subversion and git services commented on CLOUDSTACK-9238:
-

Commit f79b3e1c2ad747d2aea6530c6672737915af13ab in cloudstack's branch 
refs/heads/master from [~remibergsma]
[ https://git-wip-us.apache.org/repos/asf?p=cloudstack.git;h=f79b3e1 ]

Merge pull request #1341 from wido/CLOUDSTACK-9238

CLOUDSTACK-9238: Increase URL fields to 2048 charachters from 255255 characters 
is to small for various URLs like S3 pre-signed URLs.

This causes one or more characters to be chopped of the end of the URL
and this renders them useless.

Internally in the code all URLs are passed as Strings and they are not
sized limited. This was purely in the database.

Other URL fields in the database were already 2048 characters.

This limit was introduced in the 4.1 to 4.2 upgrade when Object storage
like S3 and Swift was introduced in CloudStack for Secondary Storage.

* pr/1341:
  CLOUDSTACK-9238: Increase URL fields to 2048 charachters from 255

Signed-off-by: Remi Bergsma 


> URL fields in database are to small. Cause malformed URLs
> -
>
> Key: CLOUDSTACK-9238
> URL: https://issues.apache.org/jira/browse/CLOUDSTACK-9238
> Project: CloudStack
>  Issue Type: Bug
>  Security Level: Public(Anyone can view this level - this is the 
> default.) 
>  Components: Management Server
>Affects Versions: 4.6.0, 4.7.0, 4.8.0
>Reporter: Wido den Hollander
>Assignee: Wido den Hollander
>  Labels: s3, template, templates, url, urls
> Fix For: 4.8.0
>
>
> In for example the 'volume_store_ref' table the 'download_url' field is a 
> VARCHAR(255).
> This is to short for URLs. Same goes for the 'object_datastore_ref' table and 
> the 'url' field.
> With pre-signed S3 URLs this will cause issues.
> The fix would be to change the database fields to a VARCHAR(2048). No 
> code-change is required.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (CLOUDSTACK-9238) URL fields in database are to small. Cause malformed URLs

[ASF subversion and git services (JIRA)]

[ 
https://issues.apache.org/jira/browse/CLOUDSTACK-9238?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15103315#comment-15103315
 ] 

ASF subversion and git services commented on CLOUDSTACK-9238:
-

Commit f79b3e1c2ad747d2aea6530c6672737915af13ab in cloudstack's branch 
refs/heads/master from [~remibergsma]
[ https://git-wip-us.apache.org/repos/asf?p=cloudstack.git;h=f79b3e1 ]

Merge pull request #1341 from wido/CLOUDSTACK-9238

CLOUDSTACK-9238: Increase URL fields to 2048 charachters from 255255 characters 
is to small for various URLs like S3 pre-signed URLs.

This causes one or more characters to be chopped of the end of the URL
and this renders them useless.

Internally in the code all URLs are passed as Strings and they are not
sized limited. This was purely in the database.

Other URL fields in the database were already 2048 characters.

This limit was introduced in the 4.1 to 4.2 upgrade when Object storage
like S3 and Swift was introduced in CloudStack for Secondary Storage.

* pr/1341:
  CLOUDSTACK-9238: Increase URL fields to 2048 charachters from 255

Signed-off-by: Remi Bergsma 


> URL fields in database are to small. Cause malformed URLs
> -
>
> Key: CLOUDSTACK-9238
> URL: https://issues.apache.org/jira/browse/CLOUDSTACK-9238
> Project: CloudStack
>  Issue Type: Bug
>  Security Level: Public(Anyone can view this level - this is the 
> default.) 
>  Components: Management Server
>Affects Versions: 4.6.0, 4.7.0, 4.8.0
>Reporter: Wido den Hollander
>Assignee: Wido den Hollander
>  Labels: s3, template, templates, url, urls
> Fix For: 4.8.0
>
>
> In for example the 'volume_store_ref' table the 'download_url' field is a 
> VARCHAR(255).
> This is to short for URLs. Same goes for the 'object_datastore_ref' table and 
> the 'url' field.
> With pre-signed S3 URLs this will cause issues.
> The fix would be to change the database fields to a VARCHAR(2048). No 
> code-change is required.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (CLOUDSTACK-9238) URL fields in database are to small. Cause malformed URLs

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/CLOUDSTACK-9238?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15103445#comment-15103445
 ] 

ASF GitHub Bot commented on CLOUDSTACK-9238:


Github user wido commented on the pull request:

https://github.com/apache/cloudstack/pull/1341#issuecomment-172259203
  
Thanks @remibergsma !

That is the same reasoning as @kevindierkx and me had while looking into 
this. We went for 2k for now as it seems to match almost 100% of the use cases.


> URL fields in database are to small. Cause malformed URLs
> -
>
> Key: CLOUDSTACK-9238
> URL: https://issues.apache.org/jira/browse/CLOUDSTACK-9238
> Project: CloudStack
>  Issue Type: Bug
>  Security Level: Public(Anyone can view this level - this is the 
> default.) 
>  Components: Management Server
>Affects Versions: 4.6.0, 4.7.0, 4.8.0
>Reporter: Wido den Hollander
>Assignee: Wido den Hollander
>  Labels: s3, template, templates, url, urls
> Fix For: 4.8.0
>
>
> In for example the 'volume_store_ref' table the 'download_url' field is a 
> VARCHAR(255).
> This is to short for URLs. Same goes for the 'object_datastore_ref' table and 
> the 'url' field.
> With pre-signed S3 URLs this will cause issues.
> The fix would be to change the database fields to a VARCHAR(2048). No 
> code-change is required.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (CLOUDSTACK-9238) URL fields in database are to small. Cause malformed URLs

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/CLOUDSTACK-9238?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15101791#comment-15101791
 ] 

ASF GitHub Bot commented on CLOUDSTACK-9238:


Github user bhaisaab commented on the pull request:

https://github.com/apache/cloudstack/pull/1341#issuecomment-171967145
  
LGTM


> URL fields in database are to small. Cause malformed URLs
> -
>
> Key: CLOUDSTACK-9238
> URL: https://issues.apache.org/jira/browse/CLOUDSTACK-9238
> Project: CloudStack
>  Issue Type: Bug
>  Security Level: Public(Anyone can view this level - this is the 
> default.) 
>  Components: Management Server
>Affects Versions: 4.6.0, 4.7.0, 4.8.0
>Reporter: Wido den Hollander
>Assignee: Wido den Hollander
>  Labels: s3, template, templates, url, urls
> Fix For: 4.8.0
>
>
> In for example the 'volume_store_ref' table the 'download_url' field is a 
> VARCHAR(255).
> This is to short for URLs. Same goes for the 'object_datastore_ref' table and 
> the 'url' field.
> With pre-signed S3 URLs this will cause issues.
> The fix would be to change the database fields to a VARCHAR(2048). No 
> code-change is required.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (CLOUDSTACK-9238) URL fields in database are to small. Cause malformed URLs

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/CLOUDSTACK-9238?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15098322#comment-15098322
 ] 

ASF GitHub Bot commented on CLOUDSTACK-9238:


Github user kevindierkx commented on the pull request:

https://github.com/apache/cloudstack/pull/1341#issuecomment-171685523
  
@borisroman Because VARCHAR can handle 'most common' supported URL lengths 
perfectly fine.
Most modern browsers have a limit somewhere around 2000 characters.

More information about the matter: 
http://stackoverflow.com/questions/417142/what-is-the-maximum-length-of-a-url-in-different-browsers


> URL fields in database are to small. Cause malformed URLs
> -
>
> Key: CLOUDSTACK-9238
> URL: https://issues.apache.org/jira/browse/CLOUDSTACK-9238
> Project: CloudStack
>  Issue Type: Bug
>  Security Level: Public(Anyone can view this level - this is the 
> default.) 
>  Components: Management Server
>Affects Versions: 4.6.0, 4.7.0, 4.8.0
>Reporter: Wido den Hollander
>Assignee: Wido den Hollander
>  Labels: s3, template, templates, url, urls
> Fix For: 4.8.0
>
>
> In for example the 'volume_store_ref' table the 'download_url' field is a 
> VARCHAR(255).
> This is to short for URLs. Same goes for the 'object_datastore_ref' table and 
> the 'url' field.
> With pre-signed S3 URLs this will cause issues.
> The fix would be to change the database fields to a VARCHAR(2048). No 
> code-change is required.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (CLOUDSTACK-9238) URL fields in database are to small. Cause malformed URLs

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/CLOUDSTACK-9238?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15098294#comment-15098294
 ] 

ASF GitHub Bot commented on CLOUDSTACK-9238:


Github user borisroman commented on the pull request:

https://github.com/apache/cloudstack/pull/1341#issuecomment-171681534
  
@wido @kevindierkx Why not use TEXT? Then we'll never have to worry again 
:-)


> URL fields in database are to small. Cause malformed URLs
> -
>
> Key: CLOUDSTACK-9238
> URL: https://issues.apache.org/jira/browse/CLOUDSTACK-9238
> Project: CloudStack
>  Issue Type: Bug
>  Security Level: Public(Anyone can view this level - this is the 
> default.) 
>  Components: Management Server
>Affects Versions: 4.6.0, 4.7.0, 4.8.0
>Reporter: Wido den Hollander
>Assignee: Wido den Hollander
>  Labels: s3, template, templates, url, urls
> Fix For: 4.8.0
>
>
> In for example the 'volume_store_ref' table the 'download_url' field is a 
> VARCHAR(255).
> This is to short for URLs. Same goes for the 'object_datastore_ref' table and 
> the 'url' field.
> With pre-signed S3 URLs this will cause issues.
> The fix would be to change the database fields to a VARCHAR(2048). No 
> code-change is required.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (CLOUDSTACK-9238) URL fields in database are to small. Cause malformed URLs

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/CLOUDSTACK-9238?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15098165#comment-15098165
 ] 

ASF GitHub Bot commented on CLOUDSTACK-9238:


Github user wido commented on the pull request:

https://github.com/apache/cloudstack/pull/1341#issuecomment-171657390
  
I tested this by manually applying all the SQL patches and verifying that 
all URLs are still in the database like they should be.

Also added a new template and saw that it registered as it should.


> URL fields in database are to small. Cause malformed URLs
> -
>
> Key: CLOUDSTACK-9238
> URL: https://issues.apache.org/jira/browse/CLOUDSTACK-9238
> Project: CloudStack
>  Issue Type: Bug
>  Security Level: Public(Anyone can view this level - this is the 
> default.) 
>  Components: Management Server
>Affects Versions: 4.6.0, 4.7.0, 4.8.0
>Reporter: Wido den Hollander
>Assignee: Wido den Hollander
>  Labels: s3, template, templates, url, urls
> Fix For: 4.8.0
>
>
> In for example the 'volume_store_ref' table the 'download_url' field is a 
> VARCHAR(255).
> This is to short for URLs. Same goes for the 'object_datastore_ref' table and 
> the 'url' field.
> With pre-signed S3 URLs this will cause issues.
> The fix would be to change the database fields to a VARCHAR(2048). No 
> code-change is required.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (CLOUDSTACK-9238) URL fields in database are to small. Cause malformed URLs

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/CLOUDSTACK-9238?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15098161#comment-15098161
 ] 

ASF GitHub Bot commented on CLOUDSTACK-9238:


GitHub user wido opened a pull request:

https://github.com/apache/cloudstack/pull/1341

CLOUDSTACK-9238: Increase URL fields to 2048 charachters from 255

255 characters is to small for various URLs like S3 pre-signed URLs.

This causes one or more characters to be chopped of the end of the URL
and this renders them useless.

Internally in the code all URLs are passed as Strings and they are not
sized limited. This was purely in the database.

Other URL fields in the database were already 2048 characters.

This limit was introduced in the 4.1 to 4.2 upgrade when Object storage
like S3 and Swift was introduced in CloudStack for Secondary Storage.

You can merge this pull request into a Git repository by running:

$ git pull https://github.com/wido/cloudstack CLOUDSTACK-9238

Alternatively you can review and apply these changes as the patch at:

https://github.com/apache/cloudstack/pull/1341.patch

To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:

This closes #1341


commit a171bbc96dc4c44734d6c14265347f9db791a0cb
Author: Wido den Hollander 
Date:   2016-01-14T13:06:04Z

CLOUDSTACK-9238: Increase URL fields to 2048 charachters from 255

255 characters is to small for various URLs like S3 pre-signed URLs.

This causes one or more characters to be chopped of the end of the URL
and this renders them useless.

Internally in the code all URLs are passed as Strings and they are not
sized limited. This was purely in the database.

Other URL fields in the database were already 2048 characters.

This limit was introduced in the 4.1 to 4.2 upgrade when Object storage
like S3 and Swift was introduced in CloudStack for Secondary Storage.




> URL fields in database are to small. Cause malformed URLs
> -
>
> Key: CLOUDSTACK-9238
> URL: https://issues.apache.org/jira/browse/CLOUDSTACK-9238
> Project: CloudStack
>  Issue Type: Bug
>  Security Level: Public(Anyone can view this level - this is the 
> default.) 
>  Components: Management Server
>Affects Versions: 4.6.0, 4.7.0, 4.8.0
>Reporter: Wido den Hollander
>Assignee: Wido den Hollander
>  Labels: s3, template, templates, url, urls
> Fix For: 4.8.0
>
>
> In for example the 'volume_store_ref' table the 'download_url' field is a 
> VARCHAR(255).
> This is to short for URLs. Same goes for the 'object_datastore_ref' table and 
> the 'url' field.
> With pre-signed S3 URLs this will cause issues.
> The fix would be to change the database fields to a VARCHAR(2048). No 
> code-change is required.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (CLOUDSTACK-9238) URL fields in database are to small. Cause malformed URLs

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/CLOUDSTACK-9238?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15098173#comment-15098173
 ] 

ASF GitHub Bot commented on CLOUDSTACK-9238:


Github user kevindierkx commented on the pull request:

https://github.com/apache/cloudstack/pull/1341#issuecomment-171659089
  
Should resolve problems with template extracts where the generated URL 
exceeds the 255 character limit.

BTW: 2048 is somewhere in the middle of what is defined as standard 
(website url length) and what is supported by most browsers.


> URL fields in database are to small. Cause malformed URLs
> -
>
> Key: CLOUDSTACK-9238
> URL: https://issues.apache.org/jira/browse/CLOUDSTACK-9238
> Project: CloudStack
>  Issue Type: Bug
>  Security Level: Public(Anyone can view this level - this is the 
> default.) 
>  Components: Management Server
>Affects Versions: 4.6.0, 4.7.0, 4.8.0
>Reporter: Wido den Hollander
>Assignee: Wido den Hollander
>  Labels: s3, template, templates, url, urls
> Fix For: 4.8.0
>
>
> In for example the 'volume_store_ref' table the 'download_url' field is a 
> VARCHAR(255).
> This is to short for URLs. Same goes for the 'object_datastore_ref' table and 
> the 'url' field.
> With pre-signed S3 URLs this will cause issues.
> The fix would be to change the database fields to a VARCHAR(2048). No 
> code-change is required.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (CLOUDSTACK-9238) URL fields in database are to small. Cause malformed URLs

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/CLOUDSTACK-9238?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15098398#comment-15098398
 ] 

ASF GitHub Bot commented on CLOUDSTACK-9238:


Github user wido commented on the pull request:

https://github.com/apache/cloudstack/pull/1341#issuecomment-171703995
  
Like @kevindierkx indeed says. A VARCHAR can be up to 64k characters long. 
It seems that 2k is the maximum length of a URL used in most systems.

That's why we sticked with the VARCHAR and didn't go for text. A VARCHAR is 
also a bit more optimal inside MySQL than a TEXT field.


> URL fields in database are to small. Cause malformed URLs
> -
>
> Key: CLOUDSTACK-9238
> URL: https://issues.apache.org/jira/browse/CLOUDSTACK-9238
> Project: CloudStack
>  Issue Type: Bug
>  Security Level: Public(Anyone can view this level - this is the 
> default.) 
>  Components: Management Server
>Affects Versions: 4.6.0, 4.7.0, 4.8.0
>Reporter: Wido den Hollander
>Assignee: Wido den Hollander
>  Labels: s3, template, templates, url, urls
> Fix For: 4.8.0
>
>
> In for example the 'volume_store_ref' table the 'download_url' field is a 
> VARCHAR(255).
> This is to short for URLs. Same goes for the 'object_datastore_ref' table and 
> the 'url' field.
> With pre-signed S3 URLs this will cause issues.
> The fix would be to change the database fields to a VARCHAR(2048). No 
> code-change is required.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)