Re: lower and upper offset not working in spark with mysql database
thanks
On Mon, Jul 6, 2015 at 10:46 AM, Manohar753 [via Apache Spark User List] <
ml-node+s1001560n23637...@n3.nabble.com> wrote:
> I think you should mention partitionColumn like below and the Colum type
> should be numeric. It works for my case.
>
>
>
> options.put("partitionColumn", "revision");
>
>
>
>
>
> Thanks,
>
> Manohar
>
>
>
>
>
> *From:* Hafiz Mujadid [via Apache Spark User List] [mailto:ml-node+[hidden
> email] <http:///user/SendEmail.jtp?type=node&node=23637&i=0>]
> *Sent:* Monday, July 6, 2015 10:56 AM
> *To:* Manohar Reddy
> *Subject:* lower and upper offset not working in spark with mysql database
>
>
>
> Hi all!
>
> I am trying to read records from offset 100 to 110 from a table using
> following piece of code
>
> val sc = new SparkContext(new
> SparkConf().setAppName("SparkJdbcDs").setMaster("local[*]"))
> val sqlContext = new SQLContext(sc)
> val options = new HashMap[String, String]()
> options.put("driver", "com.mysql.jdbc.Driver")
> options.put("url",
> "jdbc:mysql://***:3306/temp?user=&password=")
> options.put("dbtable", "tempTable")
> options.put("lowerBound", "100")
> options.put("upperBound", "110")
> options.put("numPartitions", "1")
> sqlContext.load("jdbc", options)
>
>
> but this returns all the records instead of only 10 records
>
> --
>
> *If you reply to this email, your message will be added to the discussion
> below:*
>
>
> http://apache-spark-user-list.1001560.n3.nabble.com/lower-and-upper-offset-not-working-in-spark-with-mysql-database-tp23635.html
>
> To start a new topic under Apache Spark User List, email [hidden email]
> <http:///user/SendEmail.jtp?type=node&node=23637&i=1>
> To unsubscribe from Apache Spark User List, click here.
> NAML
> <http://apache-spark-user-list.1001560.n3.nabble.com/template/NamlServlet.jtp?macro=macro_viewer&id=instant_html%21nabble%3Aemail.naml&base=nabble.naml.namespaces.BasicNamespace-nabble.view.web.template.NabbleNamespace-nabble.view.web.template.NodeNamespace&breadcrumbs=notify_subscribers%21nabble%3Aemail.naml-instant_emails%21nabble%3Aemail.naml-send_instant_email%21nabble%3Aemail.naml>
> --
> Happiest Minds Disclaimer
>
> This message is for the sole use of the intended recipient(s) and may
> contain confidential, proprietary or legally privileged information. Any
> unauthorized review, use, disclosure or distribution is prohibited. If you
> are not the original intended recipient of the message, please contact the
> sender by reply email and destroy all copies of the original message.
> Happiest Minds Technologies <http://www.happiestminds.com>
>
> --
>
>
> --
> If you reply to this email, your message will be added to the discussion
> below:
>
> http://apache-spark-user-list.1001560.n3.nabble.com/lower-and-upper-offset-not-working-in-spark-with-mysql-database-tp23635p23637.html
> To unsubscribe from lower and upper offset not working in spark with
> mysql database, click here
> <http://apache-spark-user-list.1001560.n3.nabble.com/template/NamlServlet.jtp?macro=unsubscribe_by_code&node=23635&code=aGFmaXptdWphZGlkMDBAZ21haWwuY29tfDIzNjM1fC05MjEzOTMxMTE=>
> .
> NAML
> <http://apache-spark-user-list.1001560.n3.nabble.com/template/NamlServlet.jtp?macro=macro_viewer&id=instant_html%21nabble%3Aemail.naml&base=nabble.naml.namespaces.BasicNamespace-nabble.view.web.template.NabbleNamespace-nabble.view.web.template.NodeNamespace&breadcrumbs=notify_subscribers%21nabble%3Aemail.naml-instant_emails%21nabble%3Aemail.naml-send_instant_email%21nabble%3Aemail.naml>
>
--
Regards: HAFIZ MUJADID
--
View this message in context:
http://apache-spark-user-list.1001560.n3.nabble.com/lower-and-upper-offset-not-working-in-spark-with-mysql-database-tp23635p23638.html
Sent from the Apache Spark User List mailing list archive at Nabble.com.
RE: lower and upper offset not working in spark with mysql database
I think you should mention partitionColumn like below and the Colum type should
be numeric. It works for my case.
options.put("partitionColumn", "revision");
Thanks,
Manohar
From: Hafiz Mujadid [via Apache Spark User List]
[mailto:ml-node+s1001560n23635...@n3.nabble.com]
Sent: Monday, July 6, 2015 10:56 AM
To: Manohar Reddy
Subject: lower and upper offset not working in spark with mysql database
Hi all!
I am trying to read records from offset 100 to 110 from a table using following
piece of code
val sc = new SparkContext(new
SparkConf().setAppName("SparkJdbcDs").setMaster("local[*]"))
val sqlContext = new SQLContext(sc)
val options = new HashMap[String, String]()
options.put("driver", "com.mysql.jdbc.Driver")
options.put("url", "jdbc:mysql://***:3306/temp?user=&password=")
options.put("dbtable", "tempTable")
options.put("lowerBound", "100")
options.put("upperBound", "110")
options.put("numPartitions", "1")
sqlContext.load("jdbc", options)
but this returns all the records instead of only 10 records
____________________
If you reply to this email, your message will be added to the discussion below:
http://apache-spark-user-list.1001560.n3.nabble.com/lower-and-upper-offset-not-working-in-spark-with-mysql-database-tp23635.html
To start a new topic under Apache Spark User List, email
ml-node+s1001560n1...@n3.nabble.com<mailto:ml-node+s1001560n1...@n3.nabble.com>
To unsubscribe from Apache Spark User List, click
here<http://apache-spark-user-list.1001560.n3.nabble.com/template/NamlServlet.jtp?macro=unsubscribe_by_code&node=1&code=bWFub2hhci5yZWRkeUBoYXBwaWVzdG1pbmRzLmNvbXwxfDM0NDk1ODYyNg==>.
NAML<http://apache-spark-user-list.1001560.n3.nabble.com/template/NamlServlet.jtp?macro=macro_viewer&id=instant_html%21nabble%3Aemail.naml&base=nabble.naml.namespaces.BasicNamespace-nabble.view.web.template.NabbleNamespace-nabble.view.web.template.NodeNamespace&breadcrumbs=notify_subscribers%21nabble%3Aemail.naml-instant_emails%21nabble%3Aemail.naml-send_instant_email%21nabble%3Aemail.naml>
Happiest Minds Disclaimer
This message is for the sole use of the intended recipient(s) and may contain
confidential, proprietary or legally privileged information. Any unauthorized
review, use, disclosure or distribution is prohibited. If you are not the
original intended recipient of the message, please contact the sender by reply
email and destroy all copies of the original message.
Happiest Minds Technologies <http://www.happiestminds.com>
--
View this message in context:
http://apache-spark-user-list.1001560.n3.nabble.com/lower-and-upper-offset-not-working-in-spark-with-mysql-database-tp23635p23637.html
Sent from the Apache Spark User List mailing list archive at Nabble.com.
lower and upper offset not working in spark with mysql database
Hi all!
I am trying to read records from offset 100 to 110 from a table using
following piece of code
val sc = new SparkContext(new
SparkConf().setAppName("SparkJdbcDs").setMaster("local[*]"))
val sqlContext = new SQLContext(sc)
val options = new HashMap[String, String]()
options.put("driver", "com.mysql.jdbc.Driver")
options.put("url",
"jdbc:mysql://***:3306/temp?user=&password=")
options.put("dbtable", "tempTable")
options.put("lowerBound", "100")
options.put("upperBound", "110")
options.put("numPartitions", "1")
sqlContext.load("jdbc", options)
but this returns all the records instead of only 10 records
--
View this message in context:
http://apache-spark-user-list.1001560.n3.nabble.com/lower-and-upper-offset-not-working-in-spark-with-mysql-database-tp23635.html
Sent from the Apache Spark User List mailing list archive at Nabble.com.
-
To unsubscribe, e-mail: user-unsubscr...@spark.apache.org
For additional commands, e-mail: user-h...@spark.apache.org
