[jira] [Commented] (TRAFODION-1734) Improvements to licensing information

[Roberta Marton (JIRA)]

[ 
https://issues.apache.org/jira/browse/TRAFODION-1734?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15083275#comment-15083275
 ] 

Roberta Marton commented on TRAFODION-1734:
---

Other things:

  remove .txt from LICENSE, README, and NOTICE files

> Improvements to licensing information 
> --
>
> Key: TRAFODION-1734
> URL: https://issues.apache.org/jira/browse/TRAFODION-1734
> Project: Apache Trafodion
>  Issue Type: Improvement
>  Components: Build Infrastructure
>Reporter: Roberta Marton
>Assignee: Roberta Marton
>
> The following suggestions were made to make licensing information more 
> fail-proof and easier to understand.
> - include type of license for BSD and MIT licenses in the LICENSE.txt file
> - make a copy of the license text for Protocol Buffer Java, jQuery 
> Foundation, and asciidocs instead of referencing an external site.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (TRAFODION-1736) REST server reports duplicate DCS connections

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/TRAFODION-1736?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15083523#comment-15083523
 ] 

ASF GitHub Bot commented on TRAFODION-1736:
---

GitHub user venkat1m opened a pull request:

https://github.com/apache/incubator-trafodion/pull/247

TRAFODION-1736 REST server reports duplicate DCS connections

Fix for TRAFODION-1736 where REST server was reporting duplicate entries 
for servers after a dcs restart.

Also fixed these other errors reported by eclipse:
- Fixed the java package name for the GetStatusResponse.java and 
ConfTool.java
- Removed the @override annotation for some of the methods which were 
either implementing interface methods or abstract methods with no real 
implementation in the super class. Touched classes are ZkClient , 
JAXBContextResolver and PlainTextMessageBodyProducer

You can merge this pull request into a Git repository by running:

$ git pull https://github.com/venkat1m/incubator-trafodion TRAFODION-1736

Alternatively you can review and apply these changes as the patch at:

https://github.com/apache/incubator-trafodion/pull/247.patch

To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:

This closes #247


commit a204166570f235be8913c93b7a359d3fc0f004b1
Author: venkat1m 
Date:   2016-01-05T18:21:23Z

TRAFODION-1736 REST server reports duplicate DCS connections




> REST server reports duplicate DCS connections
> -
>
> Key: TRAFODION-1736
> URL: https://issues.apache.org/jira/browse/TRAFODION-1736
> Project: Apache Trafodion
>  Issue Type: Bug
>  Components: connectivity-general
> Environment: DCS server is restarted using dcsstop/dcsstart but REST 
> server is not restarted.
>Reporter: Venkat Muthuswamy
>Assignee: Venkat Muthuswamy
>
> IF the DCS is restarted without restarting REST server, the REST reports 
> duplicate DCS connections. 
>  http:///v1/servers/dcs/connections  
> If you do a normal sqstop and sqrestart, everything works fine. If you also 
> bounce rest server, again everything is fine.
> The issue is when you bounce DCS only.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Comment Edited] (TRAFODION-1473) Adding external buffer and file input and extract support for LOB datatypes

[Sandhya Sundaresan (JIRA)]

[ 
https://issues.apache.org/jira/browse/TRAFODION-1473?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15067084#comment-15067084
 ] 

Sandhya Sundaresan edited comment on TRAFODION-1473 at 1/5/16 6:53 PM:
---

The JIRA has to be reopened since the the lob to buffer extract support needs 
to be enhanced to allow for cursor-like support. The caller should be able to 
specify a buffer to extract the lob data out in chunks of a specified length. 
ALso support for input from a user buffer to a lob column via insert and update 
needs to be supported.


was (Author: sandhya):
The JIRA has to be reopened since the the lob to buffer extract support needs 
to be enhanced to allow for cursor-like support. The caller should be able to 
specify a buffer to extract the lob data out in chunks of a specified length. 

> Adding external buffer and file input and extract support  for  LOB datatypes
> -
>
> Key: TRAFODION-1473
> URL: https://issues.apache.org/jira/browse/TRAFODION-1473
> Project: Apache Trafodion
>  Issue Type: Task
>  Components: sql-cmp, sql-exe
> Environment: Trafodion
>Reporter: Sandhya Sundaresan
>Assignee: Sandhya Sundaresan
>  Labels: features
> Fix For: 1.2-incubating
>
>
> LOB datatype is disabled in Trafodion  until the following support is fully 
> in.
> Here are the list of tasks and items to support :
> 1. Support input of linux files (on platform)  into LOB columns.
> 2. Support extract of LOB data into linux files on platform.
> 3. Support input of hdfs files  into LOB columns.
> 4. Support extract of LOB data into hdfs files.
> 5. Support chunking to input a very large external file into LOB columns.
> 6. Use CQD concept to limit LOB max size and limit chunk size for extract 
> /input.
> TBD 
> 7. Support for  input/extract of external files off platform into LOB columns.
> 8. Support in connectivity and driver for real LOB data type.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Resolved] (TRAFODION-1712) TPCDS install_hadoop_regr_test_env fails to create multiple level directories

[Sandhya Sundaresan (JIRA)]

 [ 
https://issues.apache.org/jira/browse/TRAFODION-1712?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Sandhya Sundaresan resolved TRAFODION-1712.
---
Resolution: Fixed

> TPCDS install_hadoop_regr_test_env fails to create multiple level directories
> -
>
> Key: TRAFODION-1712
> URL: https://issues.apache.org/jira/browse/TRAFODION-1712
> Project: Apache Trafodion
>  Issue Type: Bug
>  Components: sql-general
>Affects Versions: 2.0-incubating
>Reporter: Sandhya Sundaresan
>Assignee: Sandhya Sundaresan
>
> The setup dscript that's up hdfs directories to cretae hive tables uses :
> hadoop fs -mkdir //
> This  works in some versions of hadoop and not others. Now we need to provide 
> the -p version to mkdir if creating multiple levels of directories. If not 
> the directoryies don't get created and  the data doesn't get moved to them.
> This change is needed when running this script for the first time on new 
> machines.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (TRAFODION-1724) DatabaseMetaData.getTypeInfo() doesn't work

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/TRAFODION-1724?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15085025#comment-15085025
 ] 

ASF GitHub Bot commented on TRAFODION-1724:
---

Github user selvaganesang commented on a diff in the pull request:

https://github.com/apache/incubator-trafodion/pull/241#discussion_r48929989
  
--- Diff: core/conn/jdbc_type2/native/CSrvrStmt.cpp ---
@@ -99,7 +99,7 @@ SRVR_STMT_HDL::SRVR_STMT_HDL(long inDialogueId)
 moduleName[0] = '\0';
 inputDescName[0] = '\0';
 outputDescName[0] = '\0';
-isClosed = TRUE;
+isClosed = FALSE;
--- End diff --

Changing the default value for isClosed from TRUE to FALSE can have other 
repercussions. We might miss throwing error. Can you please confirm this change.


> DatabaseMetaData.getTypeInfo() doesn't work
> ---
>
> Key: TRAFODION-1724
> URL: https://issues.apache.org/jira/browse/TRAFODION-1724
> Project: Apache Trafodion
>  Issue Type: Bug
>  Components: client-jdbc-t2
>Reporter: Kevin Xu
>Assignee: Kevin Xu
>




--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (TRAFODION-1724) DatabaseMetaData.getTypeInfo() doesn't work

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/TRAFODION-1724?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15085073#comment-15085073
 ] 

ASF GitHub Bot commented on TRAFODION-1724:
---

Github user selvaganesang commented on a diff in the pull request:

https://github.com/apache/incubator-trafodion/pull/241#discussion_r48931851
  
--- Diff: core/conn/jdbc_type2/native/SrvrCommon.cpp ---
@@ -1991,36 +1972,14 @@ short do_ExecFetchAppend(
 longtotalLength=0;
 
 // Setup module filenames for MX metadata
-if (strncmp(stmtLabel, "SQL_GETTYPEINFO",15) == 0) {
-pSrvrStmt = createSrvrStmt(dialogueId,
-stmtLabel,
-,
-"NONSTOP_SQLMX_NSK.MXCS_SCHEMA.CATANSIMXGTI",
-SQLCLI_ODBC_MODULE_VERSION,
-1234567890,
-sqlStmtType,
-false,true);
-}
-else if (strncmp(stmtLabel, "SQL_JAVA_",9) == 0) {
-pSrvrStmt = createSrvrStmt(dialogueId,
-stmtLabel,
-,
-"NONSTOP_SQLMX_NSK.MXCS_SCHEMA.CATANSIMXJAVA",
-SQLCLI_ODBC_MODULE_VERSION,
-1234567890,
-sqlStmtType,
-false,true);
-}
-else {
-pSrvrStmt = createSrvrStmt(dialogueId,
-stmtLabel,
-,
-"NONSTOP_SQLMX_NSK.MXCS_SCHEMA.CATANSIMX",
-SQLCLI_ODBC_MODULE_VERSION,
-1234567890,
-sqlStmtType,
-false,true);
-}
+pSrvrStmt = createSrvrStmt(dialogueId,
+stmtLabel,
+,
+"NONSTOP_SQLMX_NSK.MXCS_SCHEMA.CATANSIMX",
--- End diff --

This is the module name parameter. Can this point to NULL instead.


> DatabaseMetaData.getTypeInfo() doesn't work
> ---
>
> Key: TRAFODION-1724
> URL: https://issues.apache.org/jira/browse/TRAFODION-1724
> Project: Apache Trafodion
>  Issue Type: Bug
>  Components: client-jdbc-t2
>Reporter: Kevin Xu
>Assignee: Kevin Xu
>




--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (TRAFODION-1724) DatabaseMetaData.getTypeInfo() doesn't work

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/TRAFODION-1724?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15083977#comment-15083977
 ] 

ASF GitHub Bot commented on TRAFODION-1724:
---

Github user hegdean commented on a diff in the pull request:

https://github.com/apache/incubator-trafodion/pull/241#discussion_r48906759
  
--- Diff: core/conn/jdbc_type2/native/SrvrCommon.cpp ---
@@ -1991,36 +1972,14 @@ short do_ExecFetchAppend(
 longtotalLength=0;
 
 // Setup module filenames for MX metadata
-if (strncmp(stmtLabel, "SQL_GETTYPEINFO",15) == 0) {
-pSrvrStmt = createSrvrStmt(dialogueId,
-stmtLabel,
-,
-"NONSTOP_SQLMX_NSK.MXCS_SCHEMA.CATANSIMXGTI",
-SQLCLI_ODBC_MODULE_VERSION,
-1234567890,
-sqlStmtType,
-false,true);
-}
-else if (strncmp(stmtLabel, "SQL_JAVA_",9) == 0) {
-pSrvrStmt = createSrvrStmt(dialogueId,
-stmtLabel,
-,
-"NONSTOP_SQLMX_NSK.MXCS_SCHEMA.CATANSIMXJAVA",
-SQLCLI_ODBC_MODULE_VERSION,
-1234567890,
-sqlStmtType,
-false,true);
-}
-else {
-pSrvrStmt = createSrvrStmt(dialogueId,
-stmtLabel,
-,
-"NONSTOP_SQLMX_NSK.MXCS_SCHEMA.CATANSIMX",
-SQLCLI_ODBC_MODULE_VERSION,
-1234567890,
-sqlStmtType,
-false,true);
-}
+pSrvrStmt = createSrvrStmt(dialogueId,
+stmtLabel,
+,
+"NONSTOP_SQLMX_NSK.MXCS_SCHEMA.CATANSIMX",
--- End diff --

Do we still need  the MXCS_SCHEMA reference?


> DatabaseMetaData.getTypeInfo() doesn't work
> ---
>
> Key: TRAFODION-1724
> URL: https://issues.apache.org/jira/browse/TRAFODION-1724
> Project: Apache Trafodion
>  Issue Type: Bug
>  Components: client-jdbc-t2
>Reporter: Kevin Xu
>Assignee: Kevin Xu
>




--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (TRAFODION-1473) Adding external buffer and file input and extract support for LOB datatypes

[Sandhya Sundaresan (JIRA)]

[ 
https://issues.apache.org/jira/browse/TRAFODION-1473?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15083830#comment-15083830
 ] 

Sandhya Sundaresan commented on TRAFODION-1473:
---

Support has been added to support new syntax that can extract to a user 
buffer.(Support for extracting into a file already there)
The new syntax looks as follows :

extract from  (lob , LOCATION , 
SIZE 

Having the SIZE as an input/output allows the user to retrieve how much was 
actually extracted.
For eg they could input a buffer of 1000 and SIZE 1000 but get back only 750 
bytes of data.

Although this syntax is going to be used mainly by jdbc applications and 
programs, to test it via SQL interfaces (for developers only)  a test driver 
has been added which gets built as part of the build . The regression test 
executor/TEST130 has been enhanced to call the testdriver to test this 
interface. 

Support has been added to support reading from a user buffer into a 
lob.(Support for reading from a string in memory or a file is already there)

Syntax :
insert into  values (...,buffertolob(LOCATION , 
SIZE < sizeaddr>)
update  set  = buffertolob(LOCATION , 
SIZE )

> Adding external buffer and file input and extract support  for  LOB datatypes
> -
>
> Key: TRAFODION-1473
> URL: https://issues.apache.org/jira/browse/TRAFODION-1473
> Project: Apache Trafodion
>  Issue Type: Task
>  Components: sql-cmp, sql-exe
> Environment: Trafodion
>Reporter: Sandhya Sundaresan
>Assignee: Sandhya Sundaresan
>  Labels: features
> Fix For: 1.2-incubating
>
>
> LOB datatype is disabled in Trafodion  until the following support is fully 
> in.
> Here are the list of tasks and items to support :
> 1. Support input of linux files (on platform)  into LOB columns.
> 2. Support extract of LOB data into linux files on platform.
> 3. Support input of hdfs files  into LOB columns.
> 4. Support extract of LOB data into hdfs files.
> 5. Support chunking to input a very large external file into LOB columns.
> 6. Use CQD concept to limit LOB max size and limit chunk size for extract 
> /input.
> TBD 
> 7. Support for  input/extract of external files off platform into LOB columns.
> 8. Support in connectivity and driver for real LOB data type.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (TRAFODION-1729) change the coprocessor deployment method

[liu ming (JIRA)]

[ 
https://issues.apache.org/jira/browse/TRAFODION-1729?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15084500#comment-15084500
 ] 

liu ming commented on TRAFODION-1729:
-

Thanks Stack, I need to learn how to submit a patch to HBase community.
And thanks Dave, here is the list of coprocessors:
org.apache.hadoop.hbase.coprocessor.transactional.TrxRegionObserver
org.apache.hadoop.hbase.coprocessor.transactional.TrxRegionEndpoint
org.apache.hadoop.hbase.coprocessor.AggregateImplementation
in the future, there is one more:
org.apache.hadoop.hbase.coprocessor.transactional.SsccRegionEndpoint

The overhead of adding coprocessor is only involved when a Region is first 
open. And I assume there is no difference between we use addCoprocessor() java 
API to add, or we modify hbase-site.xml to add. Just two different ways to add 
coprocessor in HBase. hbase-site.xml is a global setting. In fact, it will let 
ALL hbase tables to load Trafodion coprocessors, even for those native hbase 
tables, which not created by Trafodion. With this change, we can control that 
only Trafodion tables will be equipped with these coprocessors. Yet another 
reason to do this change :-) 

> change the coprocessor deployment method
> 
>
> Key: TRAFODION-1729
> URL: https://issues.apache.org/jira/browse/TRAFODION-1729
> Project: Apache Trafodion
>  Issue Type: Improvement
>  Components: dtm
>Reporter: liu ming
>Assignee: mashengchen
>
> I have a proposal to change our current HBase coprocessor configuration 
> method. 
> There are three ways to add a coprocessor to a HBase table:
> 1.   Via editing hbase-site.xml, which will load coprocessor for ALL 
> tables (Trafodion is using this method now)
> 2.   Via HBase shell command
> 3.   Via HTableDescriptor.addCoprocessor() java API
> Trafodion now is using the first method. I proposed to use method 3, I 
> finished a prototype and test seems works well.
>  
> Here are the reasons I propose for this change:
> At present, the Trafodion installer needs to modify the hbase-site.xml and 
> then restart HBase instance for the configuration to take effect. This step 
> not only complicate the installer but also let user think Trafodion is 
> intrusive into underlying HBase system. It will be ideal if we can avoid this 
> step. Another problem: in CDH, there is a concept called ‘region server 
> group’ or something, so the settings will have to carefully handled by 
> installer to apply to all groups. As we saw recently in WebRoot deployment, 
> Trafodion failed due to this reason. All these are very error prone and 
> complicate the Trafodion installer. Once CDH or HDP changed something, 
> Trafodion may fail again.
>  
> So I spent time to investigate why we need to restart HBase in order to 
> install Trafodion.  
> As I understand, there are 3 major reasons 
> 1.   To add hbase-trx coprocessors
> 2.   To overload HRegion with TransactionalRegion
> 3.   Various configuration settings, need to check one by one.
> The first configuration can be avoided by applying my proposed change.
> The second one, I look through the TransactionalRegion.java, and find out the 
> only reason (now) is to overload the getScanner() method to be public so can 
> be invoked by the coprocessor. And there are only 1 or 2 places that API is 
> invoked in Trafodion code. I checked with Kevin and he proposed by using 
> ‘java reflection’ we can also avoid this. 
> All other configuration items to some extent look like ‘best to have’, but 
> not ‘must to have’. And I also find two config items seems never been used:
> hbase.bulkload.staging.dir /hbase-staging (Suresh can confirm, 
> but I search in all code, seems this is never used)
> hbase.regionserver.region.transactional.tlog   true (Narendra can 
> confirm, this is NEVER used, maybe a legacy config item?)
> Yes, by now, there are still some other config items seems cannot be avoided, 
> but I hope we can find some way to remove them in the future. I am not trying 
> to solve all issues right now, just want to start the effort to remove 
> unnecessary hbase reconfiguration.
> For this example, Coprocessors can be added to a table at run time, no need 
> to edit the hbase-site.xml and restart hbase. This is only the first step to 
> try to remove the deep impact to the current HBase config and restart HBase.
>  
> So I asked for your opinions about this change. If you think this is 
> necessary, I will continue to file a JIRA and fix it. 
>  
> I strongly recommend to get rid of the step of ‘modify hbase-site.xml and 
> restart your hbase’ for Trafodion installation, it should be an option , to 
> tune the system to best suit Trafodion, but should not be a forced step. To 
> be note: Apache Phoenix is also a SQL on HBase, its installation will 

[jira] [Commented] (TRAFODION-1726) DatabaseMetaData.getIndexInfo does not work

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/TRAFODION-1726?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15084425#comment-15084425
 ] 

ASF GitHub Bot commented on TRAFODION-1726:
---

Github user arvind-narain commented on a diff in the pull request:

https://github.com/apache/incubator-trafodion/pull/243#discussion_r48914979
  
--- Diff: core/conn/jdbc_type2/native/SrvrCommon.cpp ---
@@ -1634,6 +1635,53 @@ short do_ExecSMD(
 tableParam[0], inputParam[0], inputParam[1],
 inputParam[2], inputParam[3]);
 break;
+   case SQL_API_SQLSTATISTICS:
+   if ((!checkIfWildCard(catalogNm, catalogNmNoEsc) || 
!checkIfWildCard(schemaNm, schemaNmNoEsc)
+   || !checkIfWildCard(tableNm, tableNmNoEsc)) && 
!metadataId)
+   {
+   executeException->exception_nr = 
odbc_SQLSvc_GetSQLCatalogs_ParamError_exn_;
+   executeException->u.ParamError.ParamDesc = 
SQLSVC_EXCEPTION_WILDCARD_NOT_SUPPORTED;
+   
FUNCTION_RETURN_NUMERIC(EXECUTE_EXCEPTION,("EXECUTE_EXCEPTION"));
+   }
+
+   if (strcmp(catalogNm,"") == 0)
+   strcpy(tableName1,SEABASE_MD_CATALOG);
+   else
+   strcpy(tableName1, catalogNm);
+   tableParam[0] = tableName1;
+   convertWildcard(metadataId, TRUE, schemaNm, expSchemaNm);
+   convertWildcardNoEsc(metadataId, TRUE, schemaNm, 
schemaNmNoEsc);
+   convertWildcard(metadataId, TRUE, tableNm, expTableNm);
+   convertWildcardNoEsc(metadataId, TRUE, tableNm, 
tableNmNoEsc);
+   if( inputParam[5] != NULL )
+   {
+   sprintf(cunique, " and index.IS_UNIQUE=1");
+   }
+   inputParam[0] = schemaNmNoEsc;
+   inputParam[1] = expSchemaNm;
+   inputParam[2] = tableNmNoEsc;
+   inputParam[3] = expTableNm;
+   inputParam[4] = NULL;
+   snprintf((char *)sqlString->dataValue._buffer, totalSize,
+   "select "
+   "obj.CATALOG_NAME TABLE_CAT, obj.SCHEMA_NAME TABLE_SCHEM, 
obj.OBJECT_NAME TABLE_NAME,"
+   "cast((case when index.IS_UNIQUE=0 then 1 else 0 end) as 
smallint) NON_UNIQUE, "
+   "cast(NULL as varchar(32)) INDEX_QUALIFIER, 
idxobj.OBJECT_NAME INDEX_NAME, "
+   "cast(-2 as smallint) TYPE, cols.COLUMN_NUMBER 
ORDINAL_POSITION, "
+   "substr(cols.COLUMN_NAME,0,CHAR_LENGTH(cols.COLUMN_NAME)) 
COLUMN_NAME, "
+   "cast((case when keys.ORDERING=0 then 'A' else 'D' end) as 
varchar(4)) ASC_OR_DESC,"
+   "cast(-2 as int) CARDINALITY, cast(-2 as int) PAGES, 
cast(NULL as varchar(10)) FILTER_CONDITION "
+   "from TRAFODION.\"_MD_\".OBJECTS obj "
+   "left join TRAFODION.\"_MD_\".INDEXES index on 
obj.object_uid=index.BASE_TABLE_UID "
+   "left join TRAFODION.\"_MD_\".OBJECTS idxobj on 
index.index_uid=idxobj.object_UID "
+   "left join TRAFODION.\"_MD_\".COLUMNS cols on 
cols.column_name<>'SYSKEY' and idxobj.object_uid=cols.object_uid "
+   "left join TRAFODION.\"_MD_\".KEYS keys on 
cols.object_uid=keys.object_uid and cols.COLUMN_NAME=keys.COLUMN_NAME and 
keys.column_name<>'SYSKEY' "
+   "where 1=1 %s"
--- End diff --

Could this %s ( uniqueness ) be added just before FOR READ UNCOMMITTED .. ? 
or is this done for performance reason ?
 " where  
 " (obj.SCHEMA_NAME ...
 ..
 " %s FOR READ ..."



> DatabaseMetaData.getIndexInfo does not work
> ---
>
> Key: TRAFODION-1726
> URL: https://issues.apache.org/jira/browse/TRAFODION-1726
> Project: Apache Trafodion
>  Issue Type: Bug
>  Components: client-jdbc-t2
>Reporter: Kevin Xu
>Assignee: Kevin Xu
>
> DatabaseMetaData.getIndexInfo("TRAFODION", "SEABASE", "OSD28DL_Z1", false, 
> true)



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (TRAFODION-1726) DatabaseMetaData.getIndexInfo does not work

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/TRAFODION-1726?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15084432#comment-15084432
 ] 

ASF GitHub Bot commented on TRAFODION-1726:
---

Github user arvind-narain commented on a diff in the pull request:

https://github.com/apache/incubator-trafodion/pull/243#discussion_r48915523
  
--- Diff: core/conn/jdbc_type2/native/SrvrSmd.cpp ---
@@ -300,6 +301,15 @@ void odbc_SQLSvc_GetSQLCatalogs_sme_(
inputParam[2] = schemaNmNoEsc;
inputParam[3] = expSchemaNm;
inputParam[4] = NULL;
+   if(uniqueness == 0)
--- End diff --

Can you please add some comment that this is related to SQL_INDEX_UNIQUE(0) 
or use the same ?


> DatabaseMetaData.getIndexInfo does not work
> ---
>
> Key: TRAFODION-1726
> URL: https://issues.apache.org/jira/browse/TRAFODION-1726
> Project: Apache Trafodion
>  Issue Type: Bug
>  Components: client-jdbc-t2
>Reporter: Kevin Xu
>Assignee: Kevin Xu
>
> DatabaseMetaData.getIndexInfo("TRAFODION", "SEABASE", "OSD28DL_Z1", false, 
> true)



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (TRAFODION-1726) DatabaseMetaData.getIndexInfo does not work

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/TRAFODION-1726?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15084434#comment-15084434
 ] 

ASF GitHub Bot commented on TRAFODION-1726:
---

Github user arvind-narain commented on a diff in the pull request:

https://github.com/apache/incubator-trafodion/pull/243#discussion_r48915554
  
--- Diff: core/conn/jdbc_type4/build.xml ---
@@ -27,10 +27,11 @@

 

-
-
-
-
+   
--- End diff --

is this change related to this Jira ?


> DatabaseMetaData.getIndexInfo does not work
> ---
>
> Key: TRAFODION-1726
> URL: https://issues.apache.org/jira/browse/TRAFODION-1726
> Project: Apache Trafodion
>  Issue Type: Bug
>  Components: client-jdbc-t2
>Reporter: Kevin Xu
>Assignee: Kevin Xu
>
> DatabaseMetaData.getIndexInfo("TRAFODION", "SEABASE", "OSD28DL_Z1", false, 
> true)



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (TRAFODION-1726) DatabaseMetaData.getIndexInfo does not work

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/TRAFODION-1726?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15084442#comment-15084442
 ] 

ASF GitHub Bot commented on TRAFODION-1726:
---

Github user arvind-narain commented on a diff in the pull request:

https://github.com/apache/incubator-trafodion/pull/243#discussion_r48915781
  
--- Diff: core/conn/jdbc_type2/native/SrvrCommon.cpp ---
@@ -1634,6 +1635,53 @@ short do_ExecSMD(
 tableParam[0], inputParam[0], inputParam[1],
 inputParam[2], inputParam[3]);
 break;
+   case SQL_API_SQLSTATISTICS:
+   if ((!checkIfWildCard(catalogNm, catalogNmNoEsc) || 
!checkIfWildCard(schemaNm, schemaNmNoEsc)
+   || !checkIfWildCard(tableNm, tableNmNoEsc)) && 
!metadataId)
+   {
+   executeException->exception_nr = 
odbc_SQLSvc_GetSQLCatalogs_ParamError_exn_;
+   executeException->u.ParamError.ParamDesc = 
SQLSVC_EXCEPTION_WILDCARD_NOT_SUPPORTED;
+   
FUNCTION_RETURN_NUMERIC(EXECUTE_EXCEPTION,("EXECUTE_EXCEPTION"));
+   }
+
+   if (strcmp(catalogNm,"") == 0)
+   strcpy(tableName1,SEABASE_MD_CATALOG);
+   else
+   strcpy(tableName1, catalogNm);
+   tableParam[0] = tableName1;
+   convertWildcard(metadataId, TRUE, schemaNm, expSchemaNm);
+   convertWildcardNoEsc(metadataId, TRUE, schemaNm, 
schemaNmNoEsc);
+   convertWildcard(metadataId, TRUE, tableNm, expTableNm);
+   convertWildcardNoEsc(metadataId, TRUE, tableNm, 
tableNmNoEsc);
+   if( inputParam[5] != NULL )
+   {
+   sprintf(cunique, " and index.IS_UNIQUE=1");
+   }
+   inputParam[0] = schemaNmNoEsc;
+   inputParam[1] = expSchemaNm;
+   inputParam[2] = tableNmNoEsc;
+   inputParam[3] = expTableNm;
+   inputParam[4] = NULL;
+   snprintf((char *)sqlString->dataValue._buffer, totalSize,
+   "select "
+   "obj.CATALOG_NAME TABLE_CAT, obj.SCHEMA_NAME TABLE_SCHEM, 
obj.OBJECT_NAME TABLE_NAME,"
+   "cast((case when index.IS_UNIQUE=0 then 1 else 0 end) as 
smallint) NON_UNIQUE, "
+   "cast(NULL as varchar(32)) INDEX_QUALIFIER, 
idxobj.OBJECT_NAME INDEX_NAME, "
+   "cast(-2 as smallint) TYPE, cols.COLUMN_NUMBER 
ORDINAL_POSITION, "
+   "substr(cols.COLUMN_NAME,0,CHAR_LENGTH(cols.COLUMN_NAME)) 
COLUMN_NAME, "
+   "cast((case when keys.ORDERING=0 then 'A' else 'D' end) as 
varchar(4)) ASC_OR_DESC,"
+   "cast(-2 as int) CARDINALITY, cast(-2 as int) PAGES, 
cast(NULL as varchar(10)) FILTER_CONDITION "
--- End diff --

Are these defaults ok ( -2 for cardinality, pages, type ) - just checking - 
I'm sure you have checked what these should be as per the API.

The other request is to add a test case for this api and port the change to 
mxosrvr also.


> DatabaseMetaData.getIndexInfo does not work
> ---
>
> Key: TRAFODION-1726
> URL: https://issues.apache.org/jira/browse/TRAFODION-1726
> Project: Apache Trafodion
>  Issue Type: Bug
>  Components: client-jdbc-t2
>Reporter: Kevin Xu
>Assignee: Kevin Xu
>
> DatabaseMetaData.getIndexInfo("TRAFODION", "SEABASE", "OSD28DL_Z1", false, 
> true)



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (TRAFODION-1473) Adding external buffer and file input and extract support for LOB datatypes

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/TRAFODION-1473?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15084067#comment-15084067
 ] 

ASF GitHub Bot commented on TRAFODION-1473:
---

Github user DaveBirdsall commented on a diff in the pull request:

https://github.com/apache/incubator-trafodion/pull/248#discussion_r48911736
  
--- Diff: core/sql/clitest/blobtest.cpp ---
@@ -0,0 +1,255 @@
+/**
+// @@@ START COPYRIGHT @@@
+//
+// Licensed to the Apache Software Foundation (ASF) under one
+// or more contributor license agreements.  See the NOTICE file
+// distributed with this work for additional information
+// regarding copyright ownership.  The ASF licenses this file
+// to you under the Apache License, Version 2.0 (the
+// "License"); you may not use this file except in compliance
+// with the License.  You may obtain a copy of the License at
+//
+//   http://www.apache.org/licenses/LICENSE-2.0
+//
+// Unless required by applicable law or agreed to in writing,
+// software distributed under the License is distributed on an
+// "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+// KIND, either express or implied.  See the License for the
+// specific language governing permissions and limitations
+// under the License.
+//
+// @@@ END COPYRIGHT @@@
+**/
+/* -*-C++-*-

+
+*
+* File: Helper functions for use by bin/clitest.cpp
+* Description:  Test driver useing exe util cli interface
+*
+*
+*
+*

+
+*/
+#include "blobtest.h"
+
+Int32 extractLengthOfLobColumn(CliGlobals *cliglob, char *lobHandle, 
+  Int64 , 
+  char *lobColumnName, char *tableName)
+{
+  Int32 retcode = 0;
+  char * query = new char[4096];
+  ExeCliInterface cliInterface((cliglob->currContext())->exHeap(), 
(Int32)SQLCHARSETCODE_UTF8, cliglob->currContext(),NULL);
+  //Use lob handle to retrieve the lob length.
+  char lobLengthResult[200];
+  str_cpy_all(lobLengthResult," ",200);
+  Int32 lobLengthResultLen = 0;
+  str_sprintf(query,"extract loblength (lob '%s') LOCATION %Ld 
",lobHandle, );
+  retcode = 
cliInterface.executeImmediate(query,lobLengthResult,,FALSE);
+
+  delete query;
+  return retcode;
+ 
+}
+
+Int32 extractLobHandle(CliGlobals *cliglob, char *& lobHandle, 
+  char *lobColumnName, char *tableName)
+{
+  Int32 retcode = 0;
+  ExeCliInterface cliInterface((cliglob->currContext())->exHeap(), 
(Int32)SQLCHARSETCODE_UTF8, cliglob->currContext(),NULL);
+  char * query = new char[4096];
+  Int32 lobHandleLen = 0;
+  str_sprintf(query,"select %s from %s",lobColumnName,tableName);
+  
+  retcode = 
cliInterface.executeImmediate(query,lobHandle,,FALSE);
+
+  if (retcode)
+return retcode;
+  lobHandle[lobHandleLen]='\0';
+  delete query;
+  
+  return retcode;
+ 
+}
+
+Int32 extractLobToBuffer(CliGlobals *cliglob, char * lobHandle, Int64 
, 
+   char *lobColumnName, char *tableName)
+{
+  Int32 retcode = 0;
+  ExeCliInterface cliInterface((cliglob->currContext())->exHeap(), 
(Int32)SQLCHARSETCODE_UTF8, cliglob->currContext(),NULL);
+  // Extract lob data into a buffer.
+  char * query = new char [500];
+  
+  char *lobFinalBuf = new char[lengthOfLob];
+  char statusBuf[200] = {'\0'};
+  Int32 statusBufLen = 0;
+  Int64 lobExtractLen = 1000;
+  char *lobDataBuf = new char[lobExtractLen];
+  
+  str_sprintf(query,"extract lobtobuffer(lob '%s', LOCATION %Ld, SIZE %Ld) 
", lobHandle, (Int64)lobDataBuf, );
+ 
+ 
+  retcode = cliInterface.executeImmediatePrepare(query);
+  short i = 0;
+  while ((retcode != 100) && !(retcode<0))
+{
+  retcode = cliInterface.clearExecFetchClose(NULL,NULL,statusBuf, 
);
+  if (!retcode)
+   {
+   memcpy((char*)&(lobFinalBuf[i]),(char *)lobDataBuf,lobExtractLen);
+   i += lobExtractLen;
+   }
+}
+  if (retcode ==100 || retcode ==0)
+{
+  FILE * lobFileId = fopen("lob_output_file","w");
+  
+  int byteCount=fwrite(lobFinalBuf,sizeof(char),lengthOfLob, 
lobFileId);
+  cout << "Writing " << byteCount << " bytes from user buffer to file 
lob_output_file in current directory" << endl;
+
+  fclose(lobFileId);
+}
+  delete  lobFinalBuf;
+  delete query;
+  

[jira] [Commented] (TRAFODION-1473) Adding external buffer and file input and extract support for LOB datatypes

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/TRAFODION-1473?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15084084#comment-15084084
 ] 

ASF GitHub Bot commented on TRAFODION-1473:
---

Github user DaveBirdsall commented on a diff in the pull request:

https://github.com/apache/incubator-trafodion/pull/248#discussion_r48912365
  
--- Diff: core/sql/executor/ExExeUtilLoad.cpp ---
@@ -3098,19 +3119,25 @@ short ExExeUtilLobExtractTcb::work()
break;
  }
 
-   if (lobDataLen_ == 0)
+   if (lobDataOutputLen == 0)
  {
step_ = CLOSE_CURSOR_;
break;
  }
 
-   remainingBytes_ = (Lng32)lobDataLen_;
+   remainingBytes_ = (Lng32)lobDataOutputLen;
currPos_ = 0;
 
-if (lobTdb().getToType() == ComTdbExeUtilLobExtract::TO_FILE_)
-  step_ = INSERT_FROM_STRING_;
-else
-  step_ = RETURN_STRING_;
+/*if (lobTdb().getToType() == 
ComTdbExeUtilLobExtract::TO_FILE_)
+  step_ = INSERT_FROM_STRING_;*/
+if (lobTdb().getToType() == 
ComTdbExeUtilLobExtract::TO_BUFFER_)
+ {
+   str_sprintf(statusString_," Success: LOB data length returned : 
%d", lobDataOutputLen);
+  
+   //lobTdb().setExtractSizeIOAddr((Int64)());
+   memcpy((char *)lobTdb().dataExtractSizeIOAddr(), (char 
*),sizeof(Int64));
+   step_ = RETURN_STATUS_;
+ }
--- End diff --

Else we remain in the same step_? Could that cause an infinite loop?


> Adding external buffer and file input and extract support  for  LOB datatypes
> -
>
> Key: TRAFODION-1473
> URL: https://issues.apache.org/jira/browse/TRAFODION-1473
> Project: Apache Trafodion
>  Issue Type: Task
>  Components: sql-cmp, sql-exe
> Environment: Trafodion
>Reporter: Sandhya Sundaresan
>Assignee: Sandhya Sundaresan
>  Labels: features
> Fix For: 1.2-incubating
>
>
> LOB datatype is disabled in Trafodion  until the following support is fully 
> in.
> Here are the list of tasks and items to support :
> 1. Support input of linux files (on platform)  into LOB columns.
> 2. Support extract of LOB data into linux files on platform.
> 3. Support input of hdfs files  into LOB columns.
> 4. Support extract of LOB data into hdfs files.
> 5. Support chunking to input a very large external file into LOB columns.
> 6. Use CQD concept to limit LOB max size and limit chunk size for extract 
> /input.
> TBD 
> 7. Support for  input/extract of external files off platform into LOB columns.
> 8. Support in connectivity and driver for real LOB data type.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (TRAFODION-1473) Adding external buffer and file input and extract support for LOB datatypes

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/TRAFODION-1473?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15084086#comment-15084086
 ] 

ASF GitHub Bot commented on TRAFODION-1473:
---

Github user DaveBirdsall commented on a diff in the pull request:

https://github.com/apache/incubator-trafodion/pull/248#discussion_r48912379
  
--- Diff: core/sql/executor/ExExeUtilLoad.cpp ---
@@ -3098,19 +3119,25 @@ short ExExeUtilLobExtractTcb::work()
break;
  }
 
-   if (lobDataLen_ == 0)
+   if (lobDataOutputLen == 0)
  {
step_ = CLOSE_CURSOR_;
break;
  }
 
-   remainingBytes_ = (Lng32)lobDataLen_;
+   remainingBytes_ = (Lng32)lobDataOutputLen;
currPos_ = 0;
 
-if (lobTdb().getToType() == ComTdbExeUtilLobExtract::TO_FILE_)
-  step_ = INSERT_FROM_STRING_;
-else
-  step_ = RETURN_STRING_;
+/*if (lobTdb().getToType() == 
ComTdbExeUtilLobExtract::TO_FILE_)
+  step_ = INSERT_FROM_STRING_;*/
--- End diff --

Should the commented-out code be deleted?


> Adding external buffer and file input and extract support  for  LOB datatypes
> -
>
> Key: TRAFODION-1473
> URL: https://issues.apache.org/jira/browse/TRAFODION-1473
> Project: Apache Trafodion
>  Issue Type: Task
>  Components: sql-cmp, sql-exe
> Environment: Trafodion
>Reporter: Sandhya Sundaresan
>Assignee: Sandhya Sundaresan
>  Labels: features
> Fix For: 1.2-incubating
>
>
> LOB datatype is disabled in Trafodion  until the following support is fully 
> in.
> Here are the list of tasks and items to support :
> 1. Support input of linux files (on platform)  into LOB columns.
> 2. Support extract of LOB data into linux files on platform.
> 3. Support input of hdfs files  into LOB columns.
> 4. Support extract of LOB data into hdfs files.
> 5. Support chunking to input a very large external file into LOB columns.
> 6. Use CQD concept to limit LOB max size and limit chunk size for extract 
> /input.
> TBD 
> 7. Support for  input/extract of external files off platform into LOB columns.
> 8. Support in connectivity and driver for real LOB data type.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (TRAFODION-1473) Adding external buffer and file input and extract support for LOB datatypes

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/TRAFODION-1473?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15084092#comment-15084092
 ] 

ASF GitHub Bot commented on TRAFODION-1473:
---

Github user DaveBirdsall commented on a diff in the pull request:

https://github.com/apache/incubator-trafodion/pull/248#discussion_r48912603
  
--- Diff: core/sql/nskgmake/clitestdriver/Makefile~ ---
@@ -0,0 +1,35 @@
+###
--- End diff --

I'm guessing you did not intend to add this file (Makefile~) to your commit.


> Adding external buffer and file input and extract support  for  LOB datatypes
> -
>
> Key: TRAFODION-1473
> URL: https://issues.apache.org/jira/browse/TRAFODION-1473
> Project: Apache Trafodion
>  Issue Type: Task
>  Components: sql-cmp, sql-exe
> Environment: Trafodion
>Reporter: Sandhya Sundaresan
>Assignee: Sandhya Sundaresan
>  Labels: features
> Fix For: 1.2-incubating
>
>
> LOB datatype is disabled in Trafodion  until the following support is fully 
> in.
> Here are the list of tasks and items to support :
> 1. Support input of linux files (on platform)  into LOB columns.
> 2. Support extract of LOB data into linux files on platform.
> 3. Support input of hdfs files  into LOB columns.
> 4. Support extract of LOB data into hdfs files.
> 5. Support chunking to input a very large external file into LOB columns.
> 6. Use CQD concept to limit LOB max size and limit chunk size for extract 
> /input.
> TBD 
> 7. Support for  input/extract of external files off platform into LOB columns.
> 8. Support in connectivity and driver for real LOB data type.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (TRAFODION-1473) Adding external buffer and file input and extract support for LOB datatypes

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/TRAFODION-1473?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15084062#comment-15084062
 ] 

ASF GitHub Bot commented on TRAFODION-1473:
---

Github user DaveBirdsall commented on a diff in the pull request:

https://github.com/apache/incubator-trafodion/pull/248#discussion_r48911708
  
--- Diff: core/sql/clitest/blobtest.cpp ---
@@ -0,0 +1,255 @@
+/**
+// @@@ START COPYRIGHT @@@
+//
+// Licensed to the Apache Software Foundation (ASF) under one
+// or more contributor license agreements.  See the NOTICE file
+// distributed with this work for additional information
+// regarding copyright ownership.  The ASF licenses this file
+// to you under the Apache License, Version 2.0 (the
+// "License"); you may not use this file except in compliance
+// with the License.  You may obtain a copy of the License at
+//
+//   http://www.apache.org/licenses/LICENSE-2.0
+//
+// Unless required by applicable law or agreed to in writing,
+// software distributed under the License is distributed on an
+// "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+// KIND, either express or implied.  See the License for the
+// specific language governing permissions and limitations
+// under the License.
+//
+// @@@ END COPYRIGHT @@@
+**/
+/* -*-C++-*-

+
+*
+* File: Helper functions for use by bin/clitest.cpp
+* Description:  Test driver useing exe util cli interface
+*
+*
+*
+*

+
+*/
+#include "blobtest.h"
+
+Int32 extractLengthOfLobColumn(CliGlobals *cliglob, char *lobHandle, 
+  Int64 , 
+  char *lobColumnName, char *tableName)
+{
+  Int32 retcode = 0;
+  char * query = new char[4096];
+  ExeCliInterface cliInterface((cliglob->currContext())->exHeap(), 
(Int32)SQLCHARSETCODE_UTF8, cliglob->currContext(),NULL);
+  //Use lob handle to retrieve the lob length.
+  char lobLengthResult[200];
+  str_cpy_all(lobLengthResult," ",200);
+  Int32 lobLengthResultLen = 0;
+  str_sprintf(query,"extract loblength (lob '%s') LOCATION %Ld 
",lobHandle, );
+  retcode = 
cliInterface.executeImmediate(query,lobLengthResult,,FALSE);
+
+  delete query;
+  return retcode;
+ 
+}
+
+Int32 extractLobHandle(CliGlobals *cliglob, char *& lobHandle, 
+  char *lobColumnName, char *tableName)
+{
+  Int32 retcode = 0;
+  ExeCliInterface cliInterface((cliglob->currContext())->exHeap(), 
(Int32)SQLCHARSETCODE_UTF8, cliglob->currContext(),NULL);
+  char * query = new char[4096];
+  Int32 lobHandleLen = 0;
+  str_sprintf(query,"select %s from %s",lobColumnName,tableName);
+  
+  retcode = 
cliInterface.executeImmediate(query,lobHandle,,FALSE);
+
+  if (retcode)
+return retcode;
+  lobHandle[lobHandleLen]='\0';
+  delete query;
+  
+  return retcode;
+ 
+}
+
+Int32 extractLobToBuffer(CliGlobals *cliglob, char * lobHandle, Int64 
, 
+   char *lobColumnName, char *tableName)
+{
+  Int32 retcode = 0;
+  ExeCliInterface cliInterface((cliglob->currContext())->exHeap(), 
(Int32)SQLCHARSETCODE_UTF8, cliglob->currContext(),NULL);
+  // Extract lob data into a buffer.
+  char * query = new char [500];
+  
+  char *lobFinalBuf = new char[lengthOfLob];
+  char statusBuf[200] = {'\0'};
+  Int32 statusBufLen = 0;
+  Int64 lobExtractLen = 1000;
+  char *lobDataBuf = new char[lobExtractLen];
+  
+  str_sprintf(query,"extract lobtobuffer(lob '%s', LOCATION %Ld, SIZE %Ld) 
", lobHandle, (Int64)lobDataBuf, );
+ 
+ 
+  retcode = cliInterface.executeImmediatePrepare(query);
+  short i = 0;
+  while ((retcode != 100) && !(retcode<0))
+{
+  retcode = cliInterface.clearExecFetchClose(NULL,NULL,statusBuf, 
);
+  if (!retcode)
+   {
+   memcpy((char*)&(lobFinalBuf[i]),(char *)lobDataBuf,lobExtractLen);
+   i += lobExtractLen;
+   }
+}
+  if (retcode ==100 || retcode ==0)
+{
+  FILE * lobFileId = fopen("lob_output_file","w");
+  
+  int byteCount=fwrite(lobFinalBuf,sizeof(char),lengthOfLob, 
lobFileId);
+  cout << "Writing " << byteCount << " bytes from user buffer to file 
lob_output_file in current directory" << endl;
+
+  fclose(lobFileId);
+}
+  delete  lobFinalBuf;
+  delete query;
+  

[jira] [Commented] (TRAFODION-1473) Adding external buffer and file input and extract support for LOB datatypes

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/TRAFODION-1473?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15084056#comment-15084056
 ] 

ASF GitHub Bot commented on TRAFODION-1473:
---

Github user DaveBirdsall commented on a diff in the pull request:

https://github.com/apache/incubator-trafodion/pull/248#discussion_r48911574
  
--- Diff: core/sql/clitest/blobtest.cpp ---
@@ -0,0 +1,255 @@
+/**
+// @@@ START COPYRIGHT @@@
+//
+// Licensed to the Apache Software Foundation (ASF) under one
+// or more contributor license agreements.  See the NOTICE file
+// distributed with this work for additional information
+// regarding copyright ownership.  The ASF licenses this file
+// to you under the Apache License, Version 2.0 (the
+// "License"); you may not use this file except in compliance
+// with the License.  You may obtain a copy of the License at
+//
+//   http://www.apache.org/licenses/LICENSE-2.0
+//
+// Unless required by applicable law or agreed to in writing,
+// software distributed under the License is distributed on an
+// "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+// KIND, either express or implied.  See the License for the
+// specific language governing permissions and limitations
+// under the License.
+//
+// @@@ END COPYRIGHT @@@
+**/
+/* -*-C++-*-

+
+*
+* File: Helper functions for use by bin/clitest.cpp
+* Description:  Test driver useing exe util cli interface
+*
+*
+*
+*

+
+*/
+#include "blobtest.h"
+
+Int32 extractLengthOfLobColumn(CliGlobals *cliglob, char *lobHandle, 
+  Int64 , 
+  char *lobColumnName, char *tableName)
+{
+  Int32 retcode = 0;
+  char * query = new char[4096];
+  ExeCliInterface cliInterface((cliglob->currContext())->exHeap(), 
(Int32)SQLCHARSETCODE_UTF8, cliglob->currContext(),NULL);
+  //Use lob handle to retrieve the lob length.
+  char lobLengthResult[200];
+  str_cpy_all(lobLengthResult," ",200);
+  Int32 lobLengthResultLen = 0;
+  str_sprintf(query,"extract loblength (lob '%s') LOCATION %Ld 
",lobHandle, );
+  retcode = 
cliInterface.executeImmediate(query,lobLengthResult,,FALSE);
+
+  delete query;
+  return retcode;
+ 
+}
+
+Int32 extractLobHandle(CliGlobals *cliglob, char *& lobHandle, 
+  char *lobColumnName, char *tableName)
+{
+  Int32 retcode = 0;
+  ExeCliInterface cliInterface((cliglob->currContext())->exHeap(), 
(Int32)SQLCHARSETCODE_UTF8, cliglob->currContext(),NULL);
+  char * query = new char[4096];
+  Int32 lobHandleLen = 0;
+  str_sprintf(query,"select %s from %s",lobColumnName,tableName);
+  
+  retcode = 
cliInterface.executeImmediate(query,lobHandle,,FALSE);
+
+  if (retcode)
+return retcode;
--- End diff --

Leaks the query memory


> Adding external buffer and file input and extract support  for  LOB datatypes
> -
>
> Key: TRAFODION-1473
> URL: https://issues.apache.org/jira/browse/TRAFODION-1473
> Project: Apache Trafodion
>  Issue Type: Task
>  Components: sql-cmp, sql-exe
> Environment: Trafodion
>Reporter: Sandhya Sundaresan
>Assignee: Sandhya Sundaresan
>  Labels: features
> Fix For: 1.2-incubating
>
>
> LOB datatype is disabled in Trafodion  until the following support is fully 
> in.
> Here are the list of tasks and items to support :
> 1. Support input of linux files (on platform)  into LOB columns.
> 2. Support extract of LOB data into linux files on platform.
> 3. Support input of hdfs files  into LOB columns.
> 4. Support extract of LOB data into hdfs files.
> 5. Support chunking to input a very large external file into LOB columns.
> 6. Use CQD concept to limit LOB max size and limit chunk size for extract 
> /input.
> TBD 
> 7. Support for  input/extract of external files off platform into LOB columns.
> 8. Support in connectivity and driver for real LOB data type.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (TRAFODION-1473) Adding external buffer and file input and extract support for LOB datatypes

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/TRAFODION-1473?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15084071#comment-15084071
 ] 

ASF GitHub Bot commented on TRAFODION-1473:
---

Github user DaveBirdsall commented on a diff in the pull request:

https://github.com/apache/incubator-trafodion/pull/248#discussion_r48911826
  
--- Diff: core/sql/comexe/ComTdbExeUtil.cpp ---
@@ -2231,28 +2233,32 @@ ComTdbExeUtilLobExtract::ComTdbExeUtilLobExtract
 handle_(handle),
 handleLen_(handleLen),
 toType_((short)toType),
+bufAddr_(bufAddr),
+extractSizeIOAddr_(extractSizeAddr),
 lobStorageType_(lobStorageType),
 stringParam1_(stringParam1),
 stringParam2_(stringParam2),
 stringParam3_(stringParam3),
 lobHdfsServer_(lobHdfsServer),
 lobHdfsPort_(lobHdfsPort),
-rowSize_(0),
-bufSize_(0),
+totalBufSize_(0),
 flags_(0)
 {
   setNodeType(ComTdb::ex_LOB_EXTRACT);
   if ((toType_ == ExtractToType::TO_BUFFER_) || (toType_ == 
ExtractToType::TO_STRING_))
 {
-  // intparam1 contains the rowsize passed in via syntax
+  /*// intparam1 contains the rowsize passed in via syntax
--- End diff --

Typo?


> Adding external buffer and file input and extract support  for  LOB datatypes
> -
>
> Key: TRAFODION-1473
> URL: https://issues.apache.org/jira/browse/TRAFODION-1473
> Project: Apache Trafodion
>  Issue Type: Task
>  Components: sql-cmp, sql-exe
> Environment: Trafodion
>Reporter: Sandhya Sundaresan
>Assignee: Sandhya Sundaresan
>  Labels: features
> Fix For: 1.2-incubating
>
>
> LOB datatype is disabled in Trafodion  until the following support is fully 
> in.
> Here are the list of tasks and items to support :
> 1. Support input of linux files (on platform)  into LOB columns.
> 2. Support extract of LOB data into linux files on platform.
> 3. Support input of hdfs files  into LOB columns.
> 4. Support extract of LOB data into hdfs files.
> 5. Support chunking to input a very large external file into LOB columns.
> 6. Use CQD concept to limit LOB max size and limit chunk size for extract 
> /input.
> TBD 
> 7. Support for  input/extract of external files off platform into LOB columns.
> 8. Support in connectivity and driver for real LOB data type.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (TRAFODION-1473) Adding external buffer and file input and extract support for LOB datatypes

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/TRAFODION-1473?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15084064#comment-15084064
 ] 

ASF GitHub Bot commented on TRAFODION-1473:
---

Github user DaveBirdsall commented on a diff in the pull request:

https://github.com/apache/incubator-trafodion/pull/248#discussion_r48911725
  
--- Diff: core/sql/clitest/blobtest.cpp ---
@@ -0,0 +1,255 @@
+/**
+// @@@ START COPYRIGHT @@@
+//
+// Licensed to the Apache Software Foundation (ASF) under one
+// or more contributor license agreements.  See the NOTICE file
+// distributed with this work for additional information
+// regarding copyright ownership.  The ASF licenses this file
+// to you under the Apache License, Version 2.0 (the
+// "License"); you may not use this file except in compliance
+// with the License.  You may obtain a copy of the License at
+//
+//   http://www.apache.org/licenses/LICENSE-2.0
+//
+// Unless required by applicable law or agreed to in writing,
+// software distributed under the License is distributed on an
+// "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+// KIND, either express or implied.  See the License for the
+// specific language governing permissions and limitations
+// under the License.
+//
+// @@@ END COPYRIGHT @@@
+**/
+/* -*-C++-*-

+
+*
+* File: Helper functions for use by bin/clitest.cpp
+* Description:  Test driver useing exe util cli interface
+*
+*
+*
+*

+
+*/
+#include "blobtest.h"
+
+Int32 extractLengthOfLobColumn(CliGlobals *cliglob, char *lobHandle, 
+  Int64 , 
+  char *lobColumnName, char *tableName)
+{
+  Int32 retcode = 0;
+  char * query = new char[4096];
+  ExeCliInterface cliInterface((cliglob->currContext())->exHeap(), 
(Int32)SQLCHARSETCODE_UTF8, cliglob->currContext(),NULL);
+  //Use lob handle to retrieve the lob length.
+  char lobLengthResult[200];
+  str_cpy_all(lobLengthResult," ",200);
+  Int32 lobLengthResultLen = 0;
+  str_sprintf(query,"extract loblength (lob '%s') LOCATION %Ld 
",lobHandle, );
+  retcode = 
cliInterface.executeImmediate(query,lobLengthResult,,FALSE);
+
+  delete query;
+  return retcode;
+ 
+}
+
+Int32 extractLobHandle(CliGlobals *cliglob, char *& lobHandle, 
+  char *lobColumnName, char *tableName)
+{
+  Int32 retcode = 0;
+  ExeCliInterface cliInterface((cliglob->currContext())->exHeap(), 
(Int32)SQLCHARSETCODE_UTF8, cliglob->currContext(),NULL);
+  char * query = new char[4096];
+  Int32 lobHandleLen = 0;
+  str_sprintf(query,"select %s from %s",lobColumnName,tableName);
+  
+  retcode = 
cliInterface.executeImmediate(query,lobHandle,,FALSE);
+
+  if (retcode)
+return retcode;
+  lobHandle[lobHandleLen]='\0';
+  delete query;
+  
+  return retcode;
+ 
+}
+
+Int32 extractLobToBuffer(CliGlobals *cliglob, char * lobHandle, Int64 
, 
+   char *lobColumnName, char *tableName)
+{
+  Int32 retcode = 0;
+  ExeCliInterface cliInterface((cliglob->currContext())->exHeap(), 
(Int32)SQLCHARSETCODE_UTF8, cliglob->currContext(),NULL);
+  // Extract lob data into a buffer.
+  char * query = new char [500];
+  
+  char *lobFinalBuf = new char[lengthOfLob];
+  char statusBuf[200] = {'\0'};
+  Int32 statusBufLen = 0;
+  Int64 lobExtractLen = 1000;
+  char *lobDataBuf = new char[lobExtractLen];
+  
+  str_sprintf(query,"extract lobtobuffer(lob '%s', LOCATION %Ld, SIZE %Ld) 
", lobHandle, (Int64)lobDataBuf, );
+ 
+ 
+  retcode = cliInterface.executeImmediatePrepare(query);
+  short i = 0;
+  while ((retcode != 100) && !(retcode<0))
+{
+  retcode = cliInterface.clearExecFetchClose(NULL,NULL,statusBuf, 
);
+  if (!retcode)
+   {
+   memcpy((char*)&(lobFinalBuf[i]),(char *)lobDataBuf,lobExtractLen);
+   i += lobExtractLen;
+   }
+}
+  if (retcode ==100 || retcode ==0)
+{
+  FILE * lobFileId = fopen("lob_output_file","w");
+  
+  int byteCount=fwrite(lobFinalBuf,sizeof(char),lengthOfLob, 
lobFileId);
+  cout << "Writing " << byteCount << " bytes from user buffer to file 
lob_output_file in current directory" << endl;
+
+  fclose(lobFileId);
+}
+  delete  lobFinalBuf;
+  delete query;
+  

[jira] [Commented] (TRAFODION-1473) Adding external buffer and file input and extract support for LOB datatypes

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/TRAFODION-1473?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15084081#comment-15084081
 ] 

ASF GitHub Bot commented on TRAFODION-1473:
---

Github user DaveBirdsall commented on a diff in the pull request:

https://github.com/apache/incubator-trafodion/pull/248#discussion_r48912122
  
--- Diff: core/sql/executor/ExExeUtilCli.cpp ---
@@ -1048,6 +1052,77 @@ Lng32 ExeCliInterface::executeImmediateExec(const 
char * stmtStr,
   return ((cliRetcode != -1) ? cliRetcode : retcode);
 }
 
+
+Lng32 ExeCliInterface::executeImmediateExecNoDealloc(const char * stmtStr,
+  char * outputBuf,
+  Lng32 * outputBufLen,
+  NABoolean nullTerminate,
+  Int64 * rowsAffected 
+  )
+{
+  Lng32 retcode = 0;
+
+  retcode = exec();
+  if (retcode < 0)
+{
+  deallocStuff(module_, stmt_, sql_src_, input_desc_, output_desc_);
+  return retcode;
+}
+
+  retcode = fetch();
+  if (retcode < 0)
+{
+  deallocStuff(module_, stmt_, sql_src_, input_desc_, output_desc_);
+  return retcode;
+}
+
+  if ((outputBuf) &&
+  (outputBufLen))
+{
+  *outputBufLen = 0;
+  if (retcode != 100)
+   {
+ char * ptr;
+ Lng32 len;
+ getPtrAndLen(1, ptr, len);
+
+ str_cpy_all(outputBuf, ptr, len);
--- End diff --

Is there any buffer overrun checking done anywhere?


> Adding external buffer and file input and extract support  for  LOB datatypes
> -
>
> Key: TRAFODION-1473
> URL: https://issues.apache.org/jira/browse/TRAFODION-1473
> Project: Apache Trafodion
>  Issue Type: Task
>  Components: sql-cmp, sql-exe
> Environment: Trafodion
>Reporter: Sandhya Sundaresan
>Assignee: Sandhya Sundaresan
>  Labels: features
> Fix For: 1.2-incubating
>
>
> LOB datatype is disabled in Trafodion  until the following support is fully 
> in.
> Here are the list of tasks and items to support :
> 1. Support input of linux files (on platform)  into LOB columns.
> 2. Support extract of LOB data into linux files on platform.
> 3. Support input of hdfs files  into LOB columns.
> 4. Support extract of LOB data into hdfs files.
> 5. Support chunking to input a very large external file into LOB columns.
> 6. Use CQD concept to limit LOB max size and limit chunk size for extract 
> /input.
> TBD 
> 7. Support for  input/extract of external files off platform into LOB columns.
> 8. Support in connectivity and driver for real LOB data type.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (TRAFODION-1473) Adding external buffer and file input and extract support for LOB datatypes

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/TRAFODION-1473?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15084087#comment-15084087
 ] 

ASF GitHub Bot commented on TRAFODION-1473:
---

Github user DaveBirdsall commented on a diff in the pull request:

https://github.com/apache/incubator-trafodion/pull/248#discussion_r48912409
  
--- Diff: core/sql/executor/ExExeUtilLoad.cpp ---
@@ -3452,12 +3492,12 @@ short ExExeUtilFileExtractTcb::work()
  break;
 
 
-   case RETURN_STRING_:
+ /*case RETURN_STRING_:
--- End diff --

Should the commented-out code be deleted?


> Adding external buffer and file input and extract support  for  LOB datatypes
> -
>
> Key: TRAFODION-1473
> URL: https://issues.apache.org/jira/browse/TRAFODION-1473
> Project: Apache Trafodion
>  Issue Type: Task
>  Components: sql-cmp, sql-exe
> Environment: Trafodion
>Reporter: Sandhya Sundaresan
>Assignee: Sandhya Sundaresan
>  Labels: features
> Fix For: 1.2-incubating
>
>
> LOB datatype is disabled in Trafodion  until the following support is fully 
> in.
> Here are the list of tasks and items to support :
> 1. Support input of linux files (on platform)  into LOB columns.
> 2. Support extract of LOB data into linux files on platform.
> 3. Support input of hdfs files  into LOB columns.
> 4. Support extract of LOB data into hdfs files.
> 5. Support chunking to input a very large external file into LOB columns.
> 6. Use CQD concept to limit LOB max size and limit chunk size for extract 
> /input.
> TBD 
> 7. Support for  input/extract of external files off platform into LOB columns.
> 8. Support in connectivity and driver for real LOB data type.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (TRAFODION-1473) Adding external buffer and file input and extract support for LOB datatypes

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/TRAFODION-1473?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15084091#comment-15084091
 ] 

ASF GitHub Bot commented on TRAFODION-1473:
---

Github user DaveBirdsall commented on a diff in the pull request:

https://github.com/apache/incubator-trafodion/pull/248#discussion_r48912575
  
--- Diff: core/sql/nskgmake/clitest/Makefile~ ---
@@ -0,0 +1,35 @@
+###
--- End diff --

I'm guessing you did not mean to add this file (Makefile~) to your commit.


> Adding external buffer and file input and extract support  for  LOB datatypes
> -
>
> Key: TRAFODION-1473
> URL: https://issues.apache.org/jira/browse/TRAFODION-1473
> Project: Apache Trafodion
>  Issue Type: Task
>  Components: sql-cmp, sql-exe
> Environment: Trafodion
>Reporter: Sandhya Sundaresan
>Assignee: Sandhya Sundaresan
>  Labels: features
> Fix For: 1.2-incubating
>
>
> LOB datatype is disabled in Trafodion  until the following support is fully 
> in.
> Here are the list of tasks and items to support :
> 1. Support input of linux files (on platform)  into LOB columns.
> 2. Support extract of LOB data into linux files on platform.
> 3. Support input of hdfs files  into LOB columns.
> 4. Support extract of LOB data into hdfs files.
> 5. Support chunking to input a very large external file into LOB columns.
> 6. Use CQD concept to limit LOB max size and limit chunk size for extract 
> /input.
> TBD 
> 7. Support for  input/extract of external files off platform into LOB columns.
> 8. Support in connectivity and driver for real LOB data type.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)