Are there Clojure meetups in NYC?

[Lawrence Krubner]

So, I just recently got back to going out to gatherings, post-pandemic. I 
live in New York City. I logged into Meetup.com, for the first time in 3 
years, and I discovered that it is a ghost town. 99% of all the Meetups are 
gone. I am a bit heartbroken about this. It used to be there were 3 or 4 
good tech meetups every night, now there are not even 1 a week. And who 
knows how long it will be before that scene comes back? 

Anyway, I'm curious if anyone knows of any Clojure gatherings that are, 
perhaps, not listed on Meetup.com? Anything at all happening in the region, 
perhaps at the universities? 

I want to emphasize that I have no interest in online videos or webinars. 
I'm strictly asking about in-person meetings. 

-- 
You received this message because you are subscribed to the Google
Groups "Clojure" group.
To post to this group, send email to clojure@googlegroups.com
Note that posts from new members are moderated - please be patient with your 
first post.
To unsubscribe from this group, send email to
clojure+unsubscr...@googlegroups.com
For more options, visit this group at
http://groups.google.com/group/clojure?hl=en
--- 
You received this message because you are subscribed to the Google Groups 
"Clojure" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to clojure+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/clojure/87a5910d-784a-4ddf-b236-0510e6bd3545n%40googlegroups.com.

Re: Why is next.jdbc using my IP address?

[Lawrence Krubner]

Okay, this seems to have to do with characters acting as wildcards in the 
password itself. This is in the .env file:

vvv*8Ezr30R%$n?L5!

but printlin in the Clojure code outputs:

vvv*8Ezr30R%?L5!

The "$n" simply vanishes. 

Why is that? This is not a regular expression. I didn't think plain strings 
in Clojure would be interpreted like this. 

Or is this how bash interprets it? 




On Friday, March 4, 2022 at 8:06:56 AM UTC-5 Lawrence Krubner wrote:

> So, as a new way to test this, I've ssh'ed to an EC2 instance that is in 
> the same VCP as the RDS database. I upload my jar file so I can run it on 
> this EC2 instance.
>
> println the hash map at startup:
>
> {:dbtype mysql, :dbname pulsedata, :user pulseuser, :password , :host 
> pulse-data.cclr8stksfch.us-west-2.rds.amazonaws.com}
> This gets me the above error. But if I copy and paste the values and use 
> them with the mysql client:
>
> mysql -u pulseuser -h pulse-data.cclr8stksfch.us-west-2.rds.amazonaws.com 
> -p pulsedata
>
> I am able to log in. This is in the same terminal window: the MySQL 
> client, at the command line in my terminal, logs me into MySQL, but "java 
> -jar user.jar" in the same terminal window gives me this error, about me 
> being rejected. I am copying and pasting the same values for user and host 
> and password. 
>
> It has to be something about the Clojure code. 
>
>
> On Friday, March 4, 2022 at 6:35:59 AM UTC-5 ore...@orestis.gr wrote:
>
>> The error message indicates that you connect with user `pulseuser` - is 
>> that the expected user? 
>>
>> I would print out the configuration that you’re passing in to jdbc.next 
>> to be absolutely certain it contains the values you expect it does.
>>
>> When you say “cli”, do you mean a mysql client? Double check the 
>> credentials (dbname, user, password) that *it* uses. 
>>
>> Best,
>> Orestis
>>
>> On 4 Mar 2022, at 13.27, Lawrence Krubner  wrote:
>>
>> But, again, I can connect from the cli using my terminal. I'm using my 
>> Spectrum connection to the Internet in both cases. If I run the app on my 
>> laptop, or I connect from the terminal, using the CLI, then in all cases 
>> I'm connecting over my Spectrum connection to the Internet. If MySQL was 
>> going to block one, then it would block the other, yes? I think the issue 
>> must have something to do with the Clojure code, since that is the only 
>> difference. I mean, if I do "java -jar user.jar" in the same terminal 
>> window, it fails, but then in that same terminal window I can connect to 
>> RDS without a problem using the CLI. 
>>
>>
>> On Thursday, March 3, 2022 at 3:26:31 PM UTC-5 red...@gmail.com wrote:
>>
>>> That is a message from MySQL, not next.jdbc.
>>>
>>> MySQL allows you to grant permissions to a user base on the host they 
>>> are connecting from so permission denied kinds of errors include the 
>>> username and the host the users connection came from.
>>>
>>> On Thu, Mar 3, 2022, 11:18 Lawrence Krubner  
>>> wrote:
>>>
>>>> I just wrote a small app that needs to connect to a MySQL app. I was 
>>>> running it on my laptop, connecting to MySQL on the laptop, and everything 
>>>> was working fine. Then I wanted to connect to one of our test databases in 
>>>> RDS in AWS. I've a simple function that finds the environment variables 
>>>> and 
>>>> returns them as a map:
>>>>
>>>> (defn get-config
>>>>   []
>>>>   (check-config)
>>>>   {
>>>>:dbtype "mysql"
>>>>:dbname (System/getenv "DATABASE_NAME")
>>>>:user (System/getenv "DATABASE_USER")
>>>>:password (System/getenv "DATABASE_PASSWORD")
>>>>:host (System/getenv "DATABASE_HOST")
>>>>})
>>>>
>>>> This is used simply:
>>>>   ds (jdbc/get-datasource  (get-config))
>>>>
>>>> I've checked the environment values several times and they are correct. 
>>>> But when the code tries to write to the RDS database, I get: 
>>>>
>>>> *java.sql.SQLException*: *Access denied for user 
>>>> 'pulseuser'@'cpe-74-71-234-49.nyc.res.rr.com 
>>>> <http://cpe-74-71-234-49.nyc.res.rr.com/>' (using password: YES)*
>>>>
>>>> I am connecting to the Internet through Spectrum, and apparently "
>>>> rr.com" is Spectrum. I'm confu

Re: Why is next.jdbc using my IP address?

[Lawrence Krubner]

So, as a new way to test this, I've ssh'ed to an EC2 instance that is in 
the same VCP as the RDS database. I upload my jar file so I can run it on 
this EC2 instance.

println the hash map at startup:

{:dbtype mysql, :dbname pulsedata, :user pulseuser, :password , :host 
pulse-data.cclr8stksfch.us-west-2.rds.amazonaws.com}
This gets me the above error. But if I copy and paste the values and use 
them with the mysql client:

mysql -u pulseuser -h pulse-data.cclr8stksfch.us-west-2.rds.amazonaws.com 
-p pulsedata

I am able to log in. This is in the same terminal window: the MySQL client, 
at the command line in my terminal, logs me into MySQL, but "java -jar 
user.jar" in the same terminal window gives me this error, about me being 
rejected. I am copying and pasting the same values for user and host and 
password. 

It has to be something about the Clojure code. 


On Friday, March 4, 2022 at 6:35:59 AM UTC-5 ore...@orestis.gr wrote:

> The error message indicates that you connect with user `pulseuser` - is 
> that the expected user? 
>
> I would print out the configuration that you’re passing in to jdbc.next to 
> be absolutely certain it contains the values you expect it does.
>
> When you say “cli”, do you mean a mysql client? Double check the 
> credentials (dbname, user, password) that *it* uses. 
>
> Best,
> Orestis
>
> On 4 Mar 2022, at 13.27, Lawrence Krubner  wrote:
>
> But, again, I can connect from the cli using my terminal. I'm using my 
> Spectrum connection to the Internet in both cases. If I run the app on my 
> laptop, or I connect from the terminal, using the CLI, then in all cases 
> I'm connecting over my Spectrum connection to the Internet. If MySQL was 
> going to block one, then it would block the other, yes? I think the issue 
> must have something to do with the Clojure code, since that is the only 
> difference. I mean, if I do "java -jar user.jar" in the same terminal 
> window, it fails, but then in that same terminal window I can connect to 
> RDS without a problem using the CLI. 
>
>
> On Thursday, March 3, 2022 at 3:26:31 PM UTC-5 red...@gmail.com wrote:
>
>> That is a message from MySQL, not next.jdbc.
>>
>> MySQL allows you to grant permissions to a user base on the host they are 
>> connecting from so permission denied kinds of errors include the username 
>> and the host the users connection came from.
>>
>> On Thu, Mar 3, 2022, 11:18 Lawrence Krubner  
>> wrote:
>>
>>> I just wrote a small app that needs to connect to a MySQL app. I was 
>>> running it on my laptop, connecting to MySQL on the laptop, and everything 
>>> was working fine. Then I wanted to connect to one of our test databases in 
>>> RDS in AWS. I've a simple function that finds the environment variables and 
>>> returns them as a map:
>>>
>>> (defn get-config
>>>   []
>>>   (check-config)
>>>   {
>>>:dbtype "mysql"
>>>:dbname (System/getenv "DATABASE_NAME")
>>>:user (System/getenv "DATABASE_USER")
>>>:password (System/getenv "DATABASE_PASSWORD")
>>>:host (System/getenv "DATABASE_HOST")
>>>})
>>>
>>> This is used simply:
>>>   ds (jdbc/get-datasource  (get-config))
>>>
>>> I've checked the environment values several times and they are correct. 
>>> But when the code tries to write to the RDS database, I get: 
>>>
>>> *java.sql.SQLException*: *Access denied for user 
>>> 'pulseuser'@'cpe-74-71-234-49.nyc.res.rr.com 
>>> <http://cpe-74-71-234-49.nyc.res.rr.com/>' (using password: YES)*
>>>
>>> I am connecting to the Internet through Spectrum, and apparently "rr.com" 
>>> is Spectrum. I'm confused, why would next.jdbc use this info? Did I forget 
>>> some important value in the hash map? 
>>>
>>> From my laptop, I can reach the RDS database using the MySQL CLI, so 
>>> there are no problems with connectivity. I whitelisted the IP address of my 
>>> house. This is something specific about what next.jdbc needs in that hash 
>>> map, I think.
>>>
>>>
>>>
>>> -- 
>>> You received this message because you are subscribed to the Google
>>> Groups "Clojure" group.
>>> To post to this group, send email to clo...@googlegroups.com
>>> Note that posts from new members are moderated - please be patient with 
>>> your first post.
>>> To unsubscribe from this group, send email to
>>> clojure+u...@googlegr

Re: Why is next.jdbc using my IP address?

[Lawrence Krubner]

But, again, I can connect from the cli using my terminal. I'm using my 
Spectrum connection to the Internet in both cases. If I run the app on my 
laptop, or I connect from the terminal, using the CLI, then in all cases 
I'm connecting over my Spectrum connection to the Internet. If MySQL was 
going to block one, then it would block the other, yes? I think the issue 
must have something to do with the Clojure code, since that is the only 
difference. I mean, if I do "java -jar user.jar" in the same terminal 
window, it fails, but then in that same terminal window I can connect to 
RDS without a problem using the CLI. 


On Thursday, March 3, 2022 at 3:26:31 PM UTC-5 redc...@gmail.com wrote:

> That is a message from MySQL, not next.jdbc.
>
> MySQL allows you to grant permissions to a user base on the host they are 
> connecting from so permission denied kinds of errors include the username 
> and the host the users connection came from.
>
> On Thu, Mar 3, 2022, 11:18 Lawrence Krubner  wrote:
>
>> I just wrote a small app that needs to connect to a MySQL app. I was 
>> running it on my laptop, connecting to MySQL on the laptop, and everything 
>> was working fine. Then I wanted to connect to one of our test databases in 
>> RDS in AWS. I've a simple function that finds the environment variables and 
>> returns them as a map:
>>
>> (defn get-config
>>   []
>>   (check-config)
>>   {
>>:dbtype "mysql"
>>:dbname (System/getenv "DATABASE_NAME")
>>:user (System/getenv "DATABASE_USER")
>>:password (System/getenv "DATABASE_PASSWORD")
>>:host (System/getenv "DATABASE_HOST")
>>})
>>
>> This is used simply:
>>   ds (jdbc/get-datasource  (get-config))
>>
>> I've checked the environment values several times and they are correct. 
>> But when the code tries to write to the RDS database, I get: 
>>
>> *java.sql.SQLException*: *Access denied for user 
>> 'pulseuser'@'cpe-74-71-234-49.nyc.res.rr.com 
>> <http://cpe-74-71-234-49.nyc.res.rr.com>' (using password: YES)*
>>
>> I am connecting to the Internet through Spectrum, and apparently "rr.com" 
>> is Spectrum. I'm confused, why would next.jdbc use this info? Did I forget 
>> some important value in the hash map? 
>>
>> From my laptop, I can reach the RDS database using the MySQL CLI, so 
>> there are no problems with connectivity. I whitelisted the IP address of my 
>> house. This is something specific about what next.jdbc needs in that hash 
>> map, I think.
>>
>>
>> -- 
>> You received this message because you are subscribed to the Google
>> Groups "Clojure" group.
>> To post to this group, send email to clo...@googlegroups.com
>> Note that posts from new members are moderated - please be patient with 
>> your first post.
>> To unsubscribe from this group, send email to
>> clojure+u...@googlegroups.com
>> For more options, visit this group at
>> http://groups.google.com/group/clojure?hl=en
>> --- 
>> You received this message because you are subscribed to the Google Groups 
>> "Clojure" group.
>> To unsubscribe from this group and stop receiving emails from it, send an 
>> email to clojure+u...@googlegroups.com.
>> To view this discussion on the web visit 
>> https://groups.google.com/d/msgid/clojure/eadd7b66-d19d-41e8-b02d-76185de0778dn%40googlegroups.com
>>  
>> <https://groups.google.com/d/msgid/clojure/eadd7b66-d19d-41e8-b02d-76185de0778dn%40googlegroups.com?utm_medium=email&utm_source=footer>
>> .
>>
>

-- 
You received this message because you are subscribed to the Google
Groups "Clojure" group.
To post to this group, send email to clojure@googlegroups.com
Note that posts from new members are moderated - please be patient with your 
first post.
To unsubscribe from this group, send email to
clojure+unsubscr...@googlegroups.com
For more options, visit this group at
http://groups.google.com/group/clojure?hl=en
--- 
You received this message because you are subscribed to the Google Groups 
"Clojure" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to clojure+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/clojure/9e58a686-29fb-43d4-af41-2cf0345425c6n%40googlegroups.com.

Why is next.jdbc using my IP address?

[Lawrence Krubner]

I just wrote a small app that needs to connect to a MySQL app. I was 
running it on my laptop, connecting to MySQL on the laptop, and everything 
was working fine. Then I wanted to connect to one of our test databases in 
RDS in AWS. I've a simple function that finds the environment variables and 
returns them as a map:

(defn get-config
  []
  (check-config)
  {
   :dbtype "mysql"
   :dbname (System/getenv "DATABASE_NAME")
   :user (System/getenv "DATABASE_USER")
   :password (System/getenv "DATABASE_PASSWORD")
   :host (System/getenv "DATABASE_HOST")
   })

This is used simply:
  ds (jdbc/get-datasource  (get-config))

I've checked the environment values several times and they are correct. But 
when the code tries to write to the RDS database, I get: 

*java.sql.SQLException*: *Access denied for user 
'pulseuser'@'cpe-74-71-234-49.nyc.res.rr.com' (using password: YES)*

I am connecting to the Internet through Spectrum, and apparently "rr.com" 
is Spectrum. I'm confused, why would next.jdbc use this info? Did I forget 
some important value in the hash map? 

>From my laptop, I can reach the RDS database using the MySQL CLI, so there 
are no problems with connectivity. I whitelisted the IP address of my 
house. This is something specific about what next.jdbc needs in that hash 
map, I think.


-- 
You received this message because you are subscribed to the Google
Groups "Clojure" group.
To post to this group, send email to clojure@googlegroups.com
Note that posts from new members are moderated - please be patient with your 
first post.
To unsubscribe from this group, send email to
clojure+unsubscr...@googlegroups.com
For more options, visit this group at
http://groups.google.com/group/clojure?hl=en
--- 
You received this message because you are subscribed to the Google Groups 
"Clojure" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to clojure+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/clojure/eadd7b66-d19d-41e8-b02d-76185de0778dn%40googlegroups.com.

How should I debug a poorly performing small web app, using Jetty?

[lawrence . krubner]

I'm looking for advice about how to debug this. 

I wrote a small web app. It has about 1,200 lines of code. It is fairly 
standard for a Clojure app, it uses Jetty, and Compojure. It uses MongoDB.

I'm just working on my MacBook Pro right now, so there is no issue such as 
using Nginx or load balancers. 

I can start the app up, and reach it on port 8082, and see all of the pages.

However, this app does have a scheduled task that runs every 15 minutes and 
which is fairly demanding. It imports a large number of JSON files from an 
S3 bucket and stores them in MongoDB. 

This all works fine, but during the import, the app is no longer responsive 
on port 8082. The browser gets nothing back. 

My first thought was "I'm stupid because I've blocked the main thread, I've 
got too much going on." But then I thought "Wait, Jetty does not run on the 
main thread, so it shouldn't be effected by whatever I'm doing on the main 
thread." 

So now I'm thinking, I'm doing some database work on the main thread, and 
this keeps the Ring handlers from looking up information from the database, 
so there is perhaps some kind of timeout issue? Like, the handler tries to 
reach MongoDB, finds it unresponsive, and gives up after a certain amount 
of time? Or too many functions are sharing a single connection to MongoDB, 
so that is the problem.

I'm testing, in part, by moving all of the database calls to background 
threads, and giving each separate connections to MongoDB, but I'm also 
looking for advice about other problems I should look for. 



-- 
You received this message because you are subscribed to the Google
Groups "Clojure" group.
To post to this group, send email to clojure@googlegroups.com
Note that posts from new members are moderated - please be patient with your 
first post.
To unsubscribe from this group, send email to
clojure+unsubscr...@googlegroups.com
For more options, visit this group at
http://groups.google.com/group/clojure?hl=en
--- 
You received this message because you are subscribed to the Google Groups 
"Clojure" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to clojure+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: using durable-queue, works locally, get :time-out when moving to an EC2

[lawrence . krubner]

> Is the problem possibly a difference between your 
> compilation environment and your deploy env?

This seems to be proven by the fact that upgrading Java on the EC2 instance 
fixed the problem. 



On Wednesday, January 16, 2019 at 3:49:41 PM UTC-5, Chris Nuernberger wrote:
>
> Are you using aot?
>
> We have used that durable queue with 1.8.  In fact, we have a 
> compatibility later that allows you to move from the durable queue to an 
> Aws queue:
> (mostly undocumented)
>
> https://github.com/techascent/tech.queue
>
> Is the problem possibly a difference between your compilation environment 
> and your deploy env?
>
>
>
> On Wed, Jan 16, 2019, 1:29 PM  wrote:
>
>> So, I upgraded to Java 11, and now everything works. So I guess this was 
>> a version conflict. 
>>
>> Just curious, but is there a way for Factual to make durable-queue to 
>> tell Leiningen that Java 11 is necessary? 
>>
>>
>>
>> On Wednesday, January 16, 2019 at 3:17:49 PM UTC-5, lawrence...@gmail.com 
>> wrote:
>>>
>>> On the new EC2 instance, running Ubuntu:
>>>
>>> java -version
>>> openjdk version "1.8.0_191"
>>> OpenJDK Runtime Environment (build 
>>> 1.8.0_191-8u191-b12-0ubuntu0.18.04.1-b12)
>>> OpenJDK 64-Bit Server VM (build 25.191-b12, mixed mode)
>>>
>>> Is it possible this version does not have the checksum signature that 
>>> durable-queue is looking for? 
>>>
>>> I'm thinking this is some kind of version problem. 
>>>
>>>
>>>
>>>
>>>
>>>
>>> On Wednesday, January 16, 2019 at 2:50:14 PM UTC-5, 
>>> lawrence...@gmail.com wrote:

 Sorry, I'm an idiot. The real error was when I called put!

 I don't understand this error:

 INFO: java.lang.NoSuchMethodError: java.util.zip.Checksum.update([B)V 
   java.lang.NoSuchMethodError: java.util.zip.Checksum.update([B)V
  at durable_queue$checksum.invokeStatic (durable_queue.clj:64)
 durable_queue$checksum.invokePrim (durable_queue.clj:-1)
 durable_queue.TaskSlab.append_to_slab_BANG_ (durable_queue.clj:314)
 durable_queue$queues$reify__6549$slab_BANG___6570.invoke 
 (durable_queue.clj:702)
 durable_queue$queues$reify__6549$fn__6575.invoke 
 (durable_queue.clj:719)
 durable_queue$queues$reify__6549.put_BANG_ (durable_queue.clj:717)
 durable_queue$queues$reify__6549.put_BANG_ (durable_queue.clj:732)
 humongorous_nlp.core$cycle_to_database$fn__13673$fn__13693.invoke 
 (core.clj:665)
 humongorous_nlp.core$cycle_to_database$fn__13673.invoke 
 (core.clj:663)
 clojure.lang.AFn.run (AFn.java:22)
 java.util.concurrent.Executors$RunnableAdapter.call 
 (Executors.java:511)
 java.util.concurrent.FutureTask.runAndReset (FutureTask.java:308)
 
 java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$301
  
 (ScheduledThreadPoolExecutor.java:180)
 
 java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run 
 (ScheduledThreadPoolExecutor.java:294)
 java.util.concurrent.ThreadPoolExecutor.runWorker 
 (ThreadPoolExecutor.java:1149)
 java.util.concurrent.ThreadPoolExecutor$Worker.run 
 (ThreadPoolExecutor.java:624)
 java.lang.Thread.run (Thread.java:748)




 On Wednesday, January 16, 2019 at 1:47:21 PM UTC-5, 
 lawrence...@gmail.com wrote:
>
> I was away from Clojure for a year and I missed it. I am pleased to be 
> back. But I've forgotten certain common errors. I feel like this is 
> something I used to know but now I've lost the knowledge. 
>
> I'm using Factual's durable-queue to put a step inbetween the import 
> of large JSON files, and their writes to the database. This works fine on 
> my local MacBook Pro, but when I move to an EC2 instance, I'm instead 
> getting time-outs when durable-queue tries to read from the queue. 
>
> At start up the app creates a few of these workers, which run for as 
> long as the app is running:
>
> (defn worker
>   [from-topics-to-persistence-queue current-database-connection]
>   (slingshot/try+
>(loop [message (durable/take! from-topics-to-persistence-queue 
> :message 6 :timed-out!)]
>  (slingshot/try+
>   (log-seq " the message in the work function " message)
>   (when (= (type message) durable_queue.Task)
> (advance message current-database-connection))
>   (catch Object o
> (log-seq "error in worker function")
> (durable/retry! message)
> (log-seq o)))
>  (recur (durable/take! from-topics-to-persistence-queue :message 
> 6 :timed-out!)))   
>(catch Object o
>  (error o)
>  (slingshot/throw+ {
> :type worker
> :error o
> :from-topics-to-persistence-queue 
> from-topics-to-persistence-queue
>   

Re: using durable-queue, works locally, get :time-out when moving to an EC2

[lawrence . krubner]

So, I upgraded to Java 11, and now everything works. So I guess this was a 
version conflict. 

Just curious, but is there a way for Factual to make durable-queue to tell 
Leiningen that Java 11 is necessary? 



On Wednesday, January 16, 2019 at 3:17:49 PM UTC-5, lawrence...@gmail.com 
wrote:
>
> On the new EC2 instance, running Ubuntu:
>
> java -version
> openjdk version "1.8.0_191"
> OpenJDK Runtime Environment (build 
> 1.8.0_191-8u191-b12-0ubuntu0.18.04.1-b12)
> OpenJDK 64-Bit Server VM (build 25.191-b12, mixed mode)
>
> Is it possible this version does not have the checksum signature that 
> durable-queue is looking for? 
>
> I'm thinking this is some kind of version problem. 
>
>
>
>
>
>
> On Wednesday, January 16, 2019 at 2:50:14 PM UTC-5, lawrence...@gmail.com 
> wrote:
>>
>> Sorry, I'm an idiot. The real error was when I called put!
>>
>> I don't understand this error:
>>
>> INFO: java.lang.NoSuchMethodError: java.util.zip.Checksum.update([B)V 
>>   java.lang.NoSuchMethodError: java.util.zip.Checksum.update([B)V
>>  at durable_queue$checksum.invokeStatic (durable_queue.clj:64)
>> durable_queue$checksum.invokePrim (durable_queue.clj:-1)
>> durable_queue.TaskSlab.append_to_slab_BANG_ (durable_queue.clj:314)
>> durable_queue$queues$reify__6549$slab_BANG___6570.invoke 
>> (durable_queue.clj:702)
>> durable_queue$queues$reify__6549$fn__6575.invoke 
>> (durable_queue.clj:719)
>> durable_queue$queues$reify__6549.put_BANG_ (durable_queue.clj:717)
>> durable_queue$queues$reify__6549.put_BANG_ (durable_queue.clj:732)
>> humongorous_nlp.core$cycle_to_database$fn__13673$fn__13693.invoke 
>> (core.clj:665)
>> humongorous_nlp.core$cycle_to_database$fn__13673.invoke (core.clj:663)
>> clojure.lang.AFn.run (AFn.java:22)
>> java.util.concurrent.Executors$RunnableAdapter.call 
>> (Executors.java:511)
>> java.util.concurrent.FutureTask.runAndReset (FutureTask.java:308)
>> 
>> java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$301
>>  
>> (ScheduledThreadPoolExecutor.java:180)
>> 
>> java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run 
>> (ScheduledThreadPoolExecutor.java:294)
>> java.util.concurrent.ThreadPoolExecutor.runWorker 
>> (ThreadPoolExecutor.java:1149)
>> java.util.concurrent.ThreadPoolExecutor$Worker.run 
>> (ThreadPoolExecutor.java:624)
>> java.lang.Thread.run (Thread.java:748)
>>
>>
>>
>>
>> On Wednesday, January 16, 2019 at 1:47:21 PM UTC-5, lawrence...@gmail.com 
>> wrote:
>>>
>>> I was away from Clojure for a year and I missed it. I am pleased to be 
>>> back. But I've forgotten certain common errors. I feel like this is 
>>> something I used to know but now I've lost the knowledge. 
>>>
>>> I'm using Factual's durable-queue to put a step inbetween the import of 
>>> large JSON files, and their writes to the database. This works fine on my 
>>> local MacBook Pro, but when I move to an EC2 instance, I'm instead getting 
>>> time-outs when durable-queue tries to read from the queue. 
>>>
>>> At start up the app creates a few of these workers, which run for as 
>>> long as the app is running:
>>>
>>> (defn worker
>>>   [from-topics-to-persistence-queue current-database-connection]
>>>   (slingshot/try+
>>>(loop [message (durable/take! from-topics-to-persistence-queue 
>>> :message 6 :timed-out!)]
>>>  (slingshot/try+
>>>   (log-seq " the message in the work function " message)
>>>   (when (= (type message) durable_queue.Task)
>>> (advance message current-database-connection))
>>>   (catch Object o
>>> (log-seq "error in worker function")
>>> (durable/retry! message)
>>> (log-seq o)))
>>>  (recur (durable/take! from-topics-to-persistence-queue :message 
>>> 6 :timed-out!)))   
>>>(catch Object o
>>>  (error o)
>>>  (slingshot/throw+ {
>>> :type worker
>>> :error o
>>> :from-topics-to-persistence-queue 
>>> from-topics-to-persistence-queue
>>> :current-database-connection 
>>> current-database-connection
>>> }
>>>
>>>
>>> On the EC2 instance, I see in the output: 
>>>
>>> Jan 16, 2019 6:43:23 PM humongorous-nlp.core invoke
>>> INFO:  the message in the work function  
>>>   
>>> Jan 16, 2019 6:43:23 PM humongorous-nlp.core invoke
>>> INFO: 
>>>
>>> Jan 16, 2019 6:43:23 PM humongorous-nlp.core invoke
>>> INFO: :timed-out! 
>>>   
>>> I'm using an environment var to set the path to the queue:
>>>
>>> PATH_TO_DURABLE_QUEUE_S3_TO_DATABASE=/tmp/s3_to_database_queue
>>>
>>> I feel like I ran into this problem 2 years ago, but I can't recall the 
>>> solution. Is the problem specific to AWS? 
>>>
>>> An important clue, I think, is that I'm not getting an error on the 
>>> put!, only on the take! Why would that be? 
>>>
>>>
>>> --- lawrence
>>>
>>>
>>>
>>>

-- 
You received this messag

Re: using durable-queue, works locally, get :time-out when moving to an EC2

[lawrence . krubner]

On the new EC2 instance, running Ubuntu:

java -version
openjdk version "1.8.0_191"
OpenJDK Runtime Environment (build 1.8.0_191-8u191-b12-0ubuntu0.18.04.1-b12)
OpenJDK 64-Bit Server VM (build 25.191-b12, mixed mode)

Is it possible this version does not have the checksum signature that 
durable-queue is looking for? 

I'm thinking this is some kind of version problem. 






On Wednesday, January 16, 2019 at 2:50:14 PM UTC-5, lawrence...@gmail.com 
wrote:
>
> Sorry, I'm an idiot. The real error was when I called put!
>
> I don't understand this error:
>
> INFO: java.lang.NoSuchMethodError: java.util.zip.Checksum.update([B)V 
>   java.lang.NoSuchMethodError: java.util.zip.Checksum.update([B)V
>  at durable_queue$checksum.invokeStatic (durable_queue.clj:64)
> durable_queue$checksum.invokePrim (durable_queue.clj:-1)
> durable_queue.TaskSlab.append_to_slab_BANG_ (durable_queue.clj:314)
> durable_queue$queues$reify__6549$slab_BANG___6570.invoke 
> (durable_queue.clj:702)
> durable_queue$queues$reify__6549$fn__6575.invoke 
> (durable_queue.clj:719)
> durable_queue$queues$reify__6549.put_BANG_ (durable_queue.clj:717)
> durable_queue$queues$reify__6549.put_BANG_ (durable_queue.clj:732)
> humongorous_nlp.core$cycle_to_database$fn__13673$fn__13693.invoke 
> (core.clj:665)
> humongorous_nlp.core$cycle_to_database$fn__13673.invoke (core.clj:663)
> clojure.lang.AFn.run (AFn.java:22)
> java.util.concurrent.Executors$RunnableAdapter.call 
> (Executors.java:511)
> java.util.concurrent.FutureTask.runAndReset (FutureTask.java:308)
> 
> java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$301
>  
> (ScheduledThreadPoolExecutor.java:180)
> 
> java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run 
> (ScheduledThreadPoolExecutor.java:294)
> java.util.concurrent.ThreadPoolExecutor.runWorker 
> (ThreadPoolExecutor.java:1149)
> java.util.concurrent.ThreadPoolExecutor$Worker.run 
> (ThreadPoolExecutor.java:624)
> java.lang.Thread.run (Thread.java:748)
>
>
>
>
> On Wednesday, January 16, 2019 at 1:47:21 PM UTC-5, lawrence...@gmail.com 
> wrote:
>>
>> I was away from Clojure for a year and I missed it. I am pleased to be 
>> back. But I've forgotten certain common errors. I feel like this is 
>> something I used to know but now I've lost the knowledge. 
>>
>> I'm using Factual's durable-queue to put a step inbetween the import of 
>> large JSON files, and their writes to the database. This works fine on my 
>> local MacBook Pro, but when I move to an EC2 instance, I'm instead getting 
>> time-outs when durable-queue tries to read from the queue. 
>>
>> At start up the app creates a few of these workers, which run for as long 
>> as the app is running:
>>
>> (defn worker
>>   [from-topics-to-persistence-queue current-database-connection]
>>   (slingshot/try+
>>(loop [message (durable/take! from-topics-to-persistence-queue 
>> :message 6 :timed-out!)]
>>  (slingshot/try+
>>   (log-seq " the message in the work function " message)
>>   (when (= (type message) durable_queue.Task)
>> (advance message current-database-connection))
>>   (catch Object o
>> (log-seq "error in worker function")
>> (durable/retry! message)
>> (log-seq o)))
>>  (recur (durable/take! from-topics-to-persistence-queue :message 
>> 6 :timed-out!)))   
>>(catch Object o
>>  (error o)
>>  (slingshot/throw+ {
>> :type worker
>> :error o
>> :from-topics-to-persistence-queue 
>> from-topics-to-persistence-queue
>> :current-database-connection 
>> current-database-connection
>> }
>>
>>
>> On the EC2 instance, I see in the output: 
>>
>> Jan 16, 2019 6:43:23 PM humongorous-nlp.core invoke
>> INFO:  the message in the work function  
>>   
>> Jan 16, 2019 6:43:23 PM humongorous-nlp.core invoke
>> INFO: 
>>
>> Jan 16, 2019 6:43:23 PM humongorous-nlp.core invoke
>> INFO: :timed-out! 
>>   
>> I'm using an environment var to set the path to the queue:
>>
>> PATH_TO_DURABLE_QUEUE_S3_TO_DATABASE=/tmp/s3_to_database_queue
>>
>> I feel like I ran into this problem 2 years ago, but I can't recall the 
>> solution. Is the problem specific to AWS? 
>>
>> An important clue, I think, is that I'm not getting an error on the put!, 
>> only on the take! Why would that be? 
>>
>>
>> --- lawrence
>>
>>
>>
>>

-- 
You received this message because you are subscribed to the Google
Groups "Clojure" group.
To post to this group, send email to clojure@googlegroups.com
Note that posts from new members are moderated - please be patient with your 
first post.
To unsubscribe from this group, send email to
clojure+unsubscr...@googlegroups.com
For more options, visit this group at
http://groups.google.com/group/clojure?hl=en
--- 
You received this message because you are subsc

Re: using durable-queue, works locally, get :time-out when moving to an EC2

[lawrence . krubner]

Sorry, I'm an idiot. The real error was when I called put!

I don't understand this error:

INFO: java.lang.NoSuchMethodError: java.util.zip.Checksum.update([B)V 
  java.lang.NoSuchMethodError: java.util.zip.Checksum.update([B)V
 at durable_queue$checksum.invokeStatic (durable_queue.clj:64)
durable_queue$checksum.invokePrim (durable_queue.clj:-1)
durable_queue.TaskSlab.append_to_slab_BANG_ (durable_queue.clj:314)
durable_queue$queues$reify__6549$slab_BANG___6570.invoke 
(durable_queue.clj:702)
durable_queue$queues$reify__6549$fn__6575.invoke (durable_queue.clj:719)
durable_queue$queues$reify__6549.put_BANG_ (durable_queue.clj:717)
durable_queue$queues$reify__6549.put_BANG_ (durable_queue.clj:732)
humongorous_nlp.core$cycle_to_database$fn__13673$fn__13693.invoke 
(core.clj:665)
humongorous_nlp.core$cycle_to_database$fn__13673.invoke (core.clj:663)
clojure.lang.AFn.run (AFn.java:22)
java.util.concurrent.Executors$RunnableAdapter.call (Executors.java:511)
java.util.concurrent.FutureTask.runAndReset (FutureTask.java:308)

java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$301 
(ScheduledThreadPoolExecutor.java:180)

java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run 
(ScheduledThreadPoolExecutor.java:294)
java.util.concurrent.ThreadPoolExecutor.runWorker 
(ThreadPoolExecutor.java:1149)
java.util.concurrent.ThreadPoolExecutor$Worker.run 
(ThreadPoolExecutor.java:624)
java.lang.Thread.run (Thread.java:748)




On Wednesday, January 16, 2019 at 1:47:21 PM UTC-5, lawrence...@gmail.com 
wrote:
>
> I was away from Clojure for a year and I missed it. I am pleased to be 
> back. But I've forgotten certain common errors. I feel like this is 
> something I used to know but now I've lost the knowledge. 
>
> I'm using Factual's durable-queue to put a step inbetween the import of 
> large JSON files, and their writes to the database. This works fine on my 
> local MacBook Pro, but when I move to an EC2 instance, I'm instead getting 
> time-outs when durable-queue tries to read from the queue. 
>
> At start up the app creates a few of these workers, which run for as long 
> as the app is running:
>
> (defn worker
>   [from-topics-to-persistence-queue current-database-connection]
>   (slingshot/try+
>(loop [message (durable/take! from-topics-to-persistence-queue :message 
> 6 :timed-out!)]
>  (slingshot/try+
>   (log-seq " the message in the work function " message)
>   (when (= (type message) durable_queue.Task)
> (advance message current-database-connection))
>   (catch Object o
> (log-seq "error in worker function")
> (durable/retry! message)
> (log-seq o)))
>  (recur (durable/take! from-topics-to-persistence-queue :message 6 
> :timed-out!)))   
>(catch Object o
>  (error o)
>  (slingshot/throw+ {
> :type worker
> :error o
> :from-topics-to-persistence-queue 
> from-topics-to-persistence-queue
> :current-database-connection 
> current-database-connection
> }
>
>
> On the EC2 instance, I see in the output: 
>
> Jan 16, 2019 6:43:23 PM humongorous-nlp.core invoke
> INFO:  the message in the work function  
>   
> Jan 16, 2019 6:43:23 PM humongorous-nlp.core invoke
> INFO: 
>
> Jan 16, 2019 6:43:23 PM humongorous-nlp.core invoke
> INFO: :timed-out! 
>   
> I'm using an environment var to set the path to the queue:
>
> PATH_TO_DURABLE_QUEUE_S3_TO_DATABASE=/tmp/s3_to_database_queue
>
> I feel like I ran into this problem 2 years ago, but I can't recall the 
> solution. Is the problem specific to AWS? 
>
> An important clue, I think, is that I'm not getting an error on the put!, 
> only on the take! Why would that be? 
>
>
> --- lawrence
>
>
>
>

-- 
You received this message because you are subscribed to the Google
Groups "Clojure" group.
To post to this group, send email to clojure@googlegroups.com
Note that posts from new members are moderated - please be patient with your 
first post.
To unsubscribe from this group, send email to
clojure+unsubscr...@googlegroups.com
For more options, visit this group at
http://groups.google.com/group/clojure?hl=en
--- 
You received this message because you are subscribed to the Google Groups 
"Clojure" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to clojure+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

using durable-queue, works locally, get :time-out when moving to an EC2

[lawrence . krubner]

I was away from Clojure for a year and I missed it. I am pleased to be 
back. But I've forgotten certain common errors. I feel like this is 
something I used to know but now I've lost the knowledge. 

I'm using Factual's durable-queue to put a step inbetween the import of 
large JSON files, and their writes to the database. This works fine on my 
local MacBook Pro, but when I move to an EC2 instance, I'm instead getting 
time-outs when durable-queue tries to read from the queue. 

At start up the app creates a few of these workers, which run for as long 
as the app is running:

(defn worker
  [from-topics-to-persistence-queue current-database-connection]
  (slingshot/try+
   (loop [message (durable/take! from-topics-to-persistence-queue :message 
6 :timed-out!)]
 (slingshot/try+
  (log-seq " the message in the work function " message)
  (when (= (type message) durable_queue.Task)
(advance message current-database-connection))
  (catch Object o
(log-seq "error in worker function")
(durable/retry! message)
(log-seq o)))
 (recur (durable/take! from-topics-to-persistence-queue :message 6 
:timed-out!)))   
   (catch Object o
 (error o)
 (slingshot/throw+ {
:type worker
:error o
:from-topics-to-persistence-queue 
from-topics-to-persistence-queue
:current-database-connection 
current-database-connection
}
   

On the EC2 instance, I see in the output: 

Jan 16, 2019 6:43:23 PM humongorous-nlp.core invoke
INFO:  the message in the work function  
  
Jan 16, 2019 6:43:23 PM humongorous-nlp.core invoke
INFO: 

Jan 16, 2019 6:43:23 PM humongorous-nlp.core invoke
INFO: :timed-out! 
  
I'm using an environment var to set the path to the queue:

PATH_TO_DURABLE_QUEUE_S3_TO_DATABASE=/tmp/s3_to_database_queue

I feel like I ran into this problem 2 years ago, but I can't recall the 
solution. Is the problem specific to AWS? 

An important clue, I think, is that I'm not getting an error on the put!, 
only on the take! Why would that be? 


--- lawrence



-- 
You received this message because you are subscribed to the Google
Groups "Clojure" group.
To post to this group, send email to clojure@googlegroups.com
Note that posts from new members are moderated - please be patient with your 
first post.
To unsubscribe from this group, send email to
clojure+unsubscr...@googlegroups.com
For more options, visit this group at
http://groups.google.com/group/clojure?hl=en
--- 
You received this message because you are subscribed to the Google Groups 
"Clojure" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to clojure+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

How does Executors/newScheduledThreadPool know how or where to parallelize work?

[lawrence . krubner]

I guess this is more of a JVM question than a Clojure question, unless 
Clojure exerts any special magic here. I'm open to a more Clojure approach 
than what I have now. 

Someone suggested I use Executors/newScheduledThreadPool for some recurring 
work, so I set it up like this: 

(def scheduler-aggregate
  (Executors/newScheduledThreadPool 32))

at the start I call:

  (.scheduleAtFixedRate  scheduler-aggregate ^Runnable (cycle-aggregate 
to-database-queue) 1 30 TimeUnit/MINUTES)

Aside from a try/catch block (which I just removed to simplify this 
example) the inner function looks like this:

(defn- cycle-aggregate
  [to-database-queue]
  (fn []
 (let [
   transcripts (query @global-database-connection {:item-type 
:transcript :processed { operators/$exists false }})
   ]
   (doseq [x transcripts]
 (aggregate-words x)
 (set-transcript-processed  @global-database-connection x)))

The function (aggregate-words) counts up a bunch of words, doing some prep 
work for a later NLP engine, and then there is this line: 

(log "The end of aggregate-words.")))

The whole process takes about 5 minutes to run, about 300 seconds. I watch 
the database and I see the number of new records increase. About every 10 
seconds I see these words appear in the logs: 

"The end of aggregate-words."

At the end of 5 minutes, these words have appeared 30 times, one for each 
of the transcripts I'm importing. 

This seems like I've done something wrong? Since the words "The end of 
aggregate-words."
appear at roughly equal intervals, and the transcripts are all about the 
same size, it seems that all of the transcripts are being handled on one 
thread. After all, if the 30 transcripts were handled on 30 threads, I'd 
expect the 30 calls to aggregate-words would all end at roughly the same 
time, instead of sequentially. 

What else do I need to do to parallelize this work? If I call (future) 
inside of aggregate-words, would the new thread come from the pool? Is 
there a way I can call aggregate-words and make sure it runs on its own 
thread from the pool? 












-- 
You received this message because you are subscribed to the Google
Groups "Clojure" group.
To post to this group, send email to clojure@googlegroups.com
Note that posts from new members are moderated - please be patient with your 
first post.
To unsubscribe from this group, send email to
clojure+unsubscr...@googlegroups.com
For more options, visit this group at
http://groups.google.com/group/clojure?hl=en
--- 
You received this message because you are subscribed to the Google Groups 
"Clojure" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to clojure+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Why did this compile? It's an obvious syntax mistake

[lawrence . krubner]

I'm wondering why this compiled? 

I was supposed to add this to my requirements:

[environ.core :refer [env]]

Instead I added this:

[environ.core :refer [env]
   
I rebuilt this, and it compiled. Then, in the cider REPL, I tried to call 
my "start" function. I was told it didn't exist, which was a big surprise, 
since I was looking right at it. 

It took me a few minutes to track down the problem. The problem was the 
missing "]". Okay, easy to fix, but why did it compile? 



-- 
You received this message because you are subscribed to the Google
Groups "Clojure" group.
To post to this group, send email to clojure@googlegroups.com
Note that posts from new members are moderated - please be patient with your 
first post.
To unsubscribe from this group, send email to
clojure+unsubscr...@googlegroups.com
For more options, visit this group at
http://groups.google.com/group/clojure?hl=en
--- 
You received this message because you are subscribed to the Google Groups 
"Clojure" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to clojure+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: Exception: clojure.lang.ExceptionInfo thrown from the UncaughtExceptionHandler in thread "main"

[lawrence . krubner]

I did not touch the  ~/.lein/profiles.clj file. 

I got things to work by removing everything related to Timbre. I would 
guess that I had some kind of naming class between Timbre and 
clojure.tools.logging. I decided to just use clojure.tools.logging, and 
that seems to have fixed the problem. 


On Monday, December 24, 2018 at 12:41:21 PM UTC-5, Sean Corfield wrote:
>
> Did you change your ~/.lein/profiles.clj file? That’s the most common 
> cause of mysterious startup failures with Leiningen it seems. Try 
> removing/renaming that file and see if your problems go away.
>
>  
>
> If you have more of the stacktrace to share (via pastebin or a gist etc), 
> that might help folks offer suggestions.
>
>  
>
> Sean Corfield -- (970) FOR-SEAN -- (904) 302-SEAN
> An Architect's View -- http://corfield.org/
>
> "If you're not annoying somebody, you're not really alive."
> -- Margaret Atwood
>
>  
> --
> *From:* clo...@googlegroups.com   > on behalf of lawrence...@gmail.com  <
> lawrence...@gmail.com >
> *Sent:* Monday, December 24, 2018 8:39:11 AM
> *To:* Clojure
> *Subject:* Exception: clojure.lang.ExceptionInfo thrown from the 
> UncaughtExceptionHandler in thread "main" 
>  
> I'm not sure what I did, but suddenly I'm getting this:
>
> Exception: clojure.lang.ExceptionInfo thrown from the 
> UncaughtExceptionHandler in thread "main"
>
> Exception: clojure.lang.ExceptionInfo thrown from the 
> UncaughtExceptionHandler in thread "Thread-0"
>
> I had been using println to do some debugging, but then I added :
>
>  [com.taoensso/timbre "4.10.0"]
>
> I think that is the last think I did. Then I did: 
>
> lein uberjar
>
> And then I tried to run that from the command line. 
>
> I've added a bunch of logging to try to get a stacktrace, but none of this 
> is working so far: 
>
> (defn -main [& args]
>
>   (slingshot/try+
>
>;; 2017-10-04 -- see this:
>;; 
> https://stackoverflow.com/questions/28908835/ssl-peer-shut-down-incorrectly-in-java
>(System/setProperty "https.protocols" "TLSv1.1")
>
>(.addShutdownHook (Runtime/getRuntime)
>  (Thread.
>   #(do (error "addShutdownHook triggered. This app is 
> shutting down.")
>(stop
>
>;; 2017-07-12 -- stealing code from here:
>;; https://stuartsierra.com/2015/05/27/clojure-uncaught-exceptions
>;; Assuming require [clojure.tools.logging :as log]
>(Thread/setDefaultUncaughtExceptionHandler
> (reify Thread$UncaughtExceptionHandler
>   (uncaughtException [_ thread ex]
> (log "the app has died ")
> (log ex)
> (log "Uncaught background exception on thread " (.getName thread) 
> " Caught by Thread/setDefaultUncaughtExceptionHandler in core/main.")
> (System/exit 1
>
>(slingshot/try+
> (start)
> (catch Exception e
>   (stack/parse-exception e)
>   (log (str e))
>   (log "the app has died!")
>   (System/exit 1))
> (catch Throwable t
>   (log t)
>   (log "the app has died!")
>   (System/exit 1))
> (catch Object o
>   (log o)
>   (log "the app has died!")
>   (System/exit 1)
>
>
> And yet all I get is:
>
> Exception: clojure.lang.ExceptionInfo thrown from the 
> UncaughtExceptionHandler in thread "main"
>
> Exception: clojure.lang.ExceptionInfo thrown from the 
> UncaughtExceptionHandler in thread "Thread-0"
>
> Somehow the other lines that I try to print never get called. Nor my call 
> to my log function which should give me a stacktrace. 
>
> What is the obvious thing to do here?
>
>
>
>
>
>
> -- 
> You received this message because you are subscribed to the Google
> Groups "Clojure" group.
> To post to this group, send email to clo...@googlegroups.com 
> Note that posts from new members are moderated - please be patient with 
> your first post.
> To unsubscribe from this group, send email to
> clojure+u...@googlegroups.com 
> For more options, visit this group at
> http://groups.google.com/group/clojure?hl=en
> --- 
> You received this message because you are subscribed to the Google Groups 
> "Clojure" group.
> To unsubscribe from this group and stop receiving emails from it, send an 
> email to clojure+u...@googlegroups.com .
> For more options, visit https://groups.google.com/d/optout.
>

-- 
You received this message because you are subscribed to the Google
Groups "Clojure" group.
To post to this group, send email to clojure@googlegroups.com
Note that posts from new members are moderated - please be patient with your 
first post.
To unsubscribe from this group, send email to
clojure+unsubscr...@googlegroups.com
For more options, visit this group at
http://groups.google.com/group/clojure?hl=en
--- 
You received this message because you are subscribed to the Google Groups 
"Clojure" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to clojure+unsubscr...@googlegroups.com.
For more option

Exception: clojure.lang.ExceptionInfo thrown from the UncaughtExceptionHandler in thread "main"

[lawrence . krubner]

I'm not sure what I did, but suddenly I'm getting this:

Exception: clojure.lang.ExceptionInfo thrown from the 
UncaughtExceptionHandler in thread "main"

Exception: clojure.lang.ExceptionInfo thrown from the 
UncaughtExceptionHandler in thread "Thread-0"

I had been using println to do some debugging, but then I added :

 [com.taoensso/timbre "4.10.0"]

I think that is the last think I did. Then I did: 

lein uberjar

And then I tried to run that from the command line. 

I've added a bunch of logging to try to get a stacktrace, but none of this 
is working so far: 

(defn -main [& args]

  (slingshot/try+

   ;; 2017-10-04 -- see this:
   ;; 
https://stackoverflow.com/questions/28908835/ssl-peer-shut-down-incorrectly-in-java
   (System/setProperty "https.protocols" "TLSv1.1")
   
   (.addShutdownHook (Runtime/getRuntime)
 (Thread.
  #(do (error "addShutdownHook triggered. This app is 
shutting down.")
   (stop

   ;; 2017-07-12 -- stealing code from here:
   ;; https://stuartsierra.com/2015/05/27/clojure-uncaught-exceptions
   ;; Assuming require [clojure.tools.logging :as log]
   (Thread/setDefaultUncaughtExceptionHandler
(reify Thread$UncaughtExceptionHandler
  (uncaughtException [_ thread ex]
(log "the app has died ")
(log ex)
(log "Uncaught background exception on thread " (.getName thread) " 
Caught by Thread/setDefaultUncaughtExceptionHandler in core/main.")
(System/exit 1

   (slingshot/try+
(start)
(catch Exception e
  (stack/parse-exception e)
  (log (str e))
  (log "the app has died!")
  (System/exit 1))
(catch Throwable t
  (log t)
  (log "the app has died!")
  (System/exit 1))
(catch Object o
  (log o)
  (log "the app has died!")
  (System/exit 1)


And yet all I get is:

Exception: clojure.lang.ExceptionInfo thrown from the 
UncaughtExceptionHandler in thread "main"

Exception: clojure.lang.ExceptionInfo thrown from the 
UncaughtExceptionHandler in thread "Thread-0"

Somehow the other lines that I try to print never get called. Nor my call 
to my log function which should give me a stacktrace. 

What is the obvious thing to do here?






-- 
You received this message because you are subscribed to the Google
Groups "Clojure" group.
To post to this group, send email to clojure@googlegroups.com
Note that posts from new members are moderated - please be patient with your 
first post.
To unsubscribe from this group, send email to
clojure+unsubscr...@googlegroups.com
For more options, visit this group at
http://groups.google.com/group/clojure?hl=en
--- 
You received this message because you are subscribed to the Google Groups 
"Clojure" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to clojure+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: What do Clojure developers use for recurring functions, other than at-at

[lawrence . krubner]

Laurens Van Houtven, good ideas, but then I'd also have to write some code 
to catch documents that got lost when a process died while trying to fetch 
a document from S3. If I simply check every 15 minutes, and grab everything 
that has not already been stored in the database, then I automatically 
fetch documents that were dropped due to error. It seems simple. In my 
current case, there is no penalty for fetching the same document twice, but 
it is important that we avoid missing a document. 



On Monday, December 17, 2018 at 2:59:22 PM UTC-5, Laurens Van Houtven wrote:
>
> Honestly I'd use CloudWatch Timed Events to kick off a Lambda or ECS 
> Fargate job (which of course you can write in Clojure) assuming you're 
> using AWS yourself anyway. If you don't care about batching maybe even just 
> attach a Lambda to the write-to-S3 bucket itself instead of checking every 
> 15m?
>
> If you want to do it in-process, my tool of choice is ztellman's manifold: 
> https://github.com/ztellman/manifold/blob/d67a8c1b9f1268c094895d70dbbf146521f5774b/src/manifold/time.clj
>
> On Mon, Dec 17, 2018 at 1:54 PM > 
> wrote:
>
>> I'm coming back to Clojure development after a year away. This is a fast 
>> moving community and it is hard to keep up when one is not working on it 
>> full time. I'm dusting off some code I wrote 2 years ago, and trying to 
>> bring all the dependencies up to their current versions. 
>>
>> I have a function that fetches files from an AWS S3 bucket, every 15 
>> minutes. I had previously used the at-at library for this:
>>
>> https://github.com/overtone/at-at
>>
>> But at-at has not been updated in 6 years, so I assume it is abandoned. I 
>> have two questions about this:
>>
>> 1.) how else do Clojure programmers usually call recurring functionality? 
>>
>> 2.) I am ignorant about the JVM, so I'm afraid I have to ask, at a 
>> fundamental level, how does at-at work? I know that if a function calls 
>> itself recurringly, on the JVM, one eventually gets stackoverflow. So how 
>> does at-at make its magic work? 
>>
>> -- 
>> You received this message because you are subscribed to the Google
>> Groups "Clojure" group.
>> To post to this group, send email to clo...@googlegroups.com 
>> 
>> Note that posts from new members are moderated - please be patient with 
>> your first post.
>> To unsubscribe from this group, send email to
>> clojure+u...@googlegroups.com 
>> For more options, visit this group at
>> http://groups.google.com/group/clojure?hl=en
>> --- 
>> You received this message because you are subscribed to the Google Groups 
>> "Clojure" group.
>> To unsubscribe from this group and stop receiving emails from it, send an 
>> email to clojure+u...@googlegroups.com .
>> For more options, visit https://groups.google.com/d/optout.
>>
>

-- 
You received this message because you are subscribed to the Google
Groups "Clojure" group.
To post to this group, send email to clojure@googlegroups.com
Note that posts from new members are moderated - please be patient with your 
first post.
To unsubscribe from this group, send email to
clojure+unsubscr...@googlegroups.com
For more options, visit this group at
http://groups.google.com/group/clojure?hl=en
--- 
You received this message because you are subscribed to the Google Groups 
"Clojure" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to clojure+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: What do Clojure developers use for recurring functions, other than at-at

[lawrence . krubner]

James Reeves, that does sound like the right way to go. I'll do that. 


On Monday, December 17, 2018 at 3:31:01 PM UTC-5, James Reeves wrote:
>
>
> I'd use an executor:
>
>   (ns example.main
> (:import [java.util.concurrent Executors TimeUnit]))
>
>   (def scheduler
> (Executors/newScheduledThreadPool 32))
>
>   (defn fetch-files []
> (println "Fetching files...))
>
>   (defn -main []
> (.scheduleAtFixedRate scheduler ^Runnable fetch-files 15 15 
> TimeUnit/MINUTES))
>
> On Mon, 17 Dec 2018 at 20:14, > wrote:
>
>> I'm coming back to Clojure development after a year away. This is a fast 
>> moving community and it is hard to keep up when one is not working on it 
>> full time. I'm dusting off some code I wrote 2 years ago, and trying to 
>> bring all the dependencies up to their current versions. 
>>
>> I have a function that fetches files from an AWS S3 bucket, every 15 
>> minutes. I had previously used the at-at library for this:
>>
>> https://github.com/overtone/at-at
>>
>> But at-at has not been updated in 6 years, so I assume it is abandoned. I 
>> have two questions about this:
>>
>> 1.) how else do Clojure programmers usually call recurring functionality? 
>>
>> 2.) I am ignorant about the JVM, so I'm afraid I have to ask, at a 
>> fundamental level, how does at-at work? I know that if a function calls 
>> itself recurringly, on the JVM, one eventually gets stackoverflow. So how 
>> does at-at make its magic work? 
>>
>> -- 
>> You received this message because you are subscribed to the Google
>> Groups "Clojure" group.
>> To post to this group, send email to clo...@googlegroups.com 
>> 
>> Note that posts from new members are moderated - please be patient with 
>> your first post.
>> To unsubscribe from this group, send email to
>> clojure+u...@googlegroups.com 
>> For more options, visit this group at
>> http://groups.google.com/group/clojure?hl=en
>> --- 
>> You received this message because you are subscribed to the Google Groups 
>> "Clojure" group.
>> To unsubscribe from this group and stop receiving emails from it, send an 
>> email to clojure+u...@googlegroups.com .
>> For more options, visit https://groups.google.com/d/optout.
>>
>
>
> -- 
> James Reeves
> booleanknot.com
>

-- 
You received this message because you are subscribed to the Google
Groups "Clojure" group.
To post to this group, send email to clojure@googlegroups.com
Note that posts from new members are moderated - please be patient with your 
first post.
To unsubscribe from this group, send email to
clojure+unsubscr...@googlegroups.com
For more options, visit this group at
http://groups.google.com/group/clojure?hl=en
--- 
You received this message because you are subscribed to the Google Groups 
"Clojure" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to clojure+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

What do Clojure developers use for recurring functions, other than at-at

[lawrence . krubner]

I'm coming back to Clojure development after a year away. This is a fast 
moving community and it is hard to keep up when one is not working on it 
full time. I'm dusting off some code I wrote 2 years ago, and trying to 
bring all the dependencies up to their current versions. 

I have a function that fetches files from an AWS S3 bucket, every 15 
minutes. I had previously used the at-at library for this:

https://github.com/overtone/at-at

But at-at has not been updated in 6 years, so I assume it is abandoned. I 
have two questions about this:

1.) how else do Clojure programmers usually call recurring functionality? 

2.) I am ignorant about the JVM, so I'm afraid I have to ask, at a 
fundamental level, how does at-at work? I know that if a function calls 
itself recurringly, on the JVM, one eventually gets stackoverflow. So how 
does at-at make its magic work? 

-- 
You received this message because you are subscribed to the Google
Groups "Clojure" group.
To post to this group, send email to clojure@googlegroups.com
Note that posts from new members are moderated - please be patient with your 
first post.
To unsubscribe from this group, send email to
clojure+unsubscr...@googlegroups.com
For more options, visit this group at
http://groups.google.com/group/clojure?hl=en
--- 
You received this message because you are subscribed to the Google Groups 
"Clojure" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to clojure+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: Received fatal alert: protocol_version

[lawrence . krubner]

Okay, on my Macbook Pro, I fixed this problem by adding this to my 
.bash_profile:

export 
JAVA_TOOL_OPTIONS="-Djavax.net.ssl.trustAnchors=$JAVA_HOME/jre/lib/security/cacerts
 
-Djavax.net.ssl.trustStore=$JAVA_HOME/jre/lib/security/cacerts"





On Thursday, November 22, 2018 at 3:37:31 PM UTC-5, lawrence...@gmail.com 
wrote:
>
> I've been away from Clojure for a year and I'm just coming back to it. I 
> assume some of my dependencies are out of date. I'm working on a project 
> that is 90% Java with a Clojure wrapper. I tried this:
>
> lein uber
>
> I got a "protocol version" error which seems to have something to do with 
> a new level of security at Maven. 
>
> So I tried this:
>
>  LEIN_JVM_OPTS=-Dhttps.protocols=TLSv1.2   lein uber
>
> and got all this: 
>
> # A fatal error has been detected by the Java Runtime Environment:
> #
> #  SIGBUS (0xa) at pc=106510692, pid=66917, tid=123145404563456
> #
> # JRE version: 7.0-b222
> # Java VM: OpenJDK 64-Bit Server VM (21.0-b17 mixed mode bsd-amd64 
> compressed oops)
> # Problematic frame:
> # J  clojure.lang.Symbol.equals(Ljava/lang/Object;)Z
> #
> # Failed to write core dump. Core dumps have been disabled. To enable core 
> dumping, try "ulimit -c unlimited" before starting Java again
> #
> # An error report file with more information is saved as:
> # 
> /Users/sugar/projects/vegetable/repos/cortex/cortext-clj/hs_err_pid66917.log
> #
> # If you would like to submit a bug report, please visit:
> #   http://bugreport.sun.com/bugreport/crash.jsp
> #
> /usr/local/bin/lein: line 152: 66917 Abort trap: 6  
>  "$LEIN_JAVA_CMD" "${BOOTCLASSPATH[@]}" -Dfile.encoding=UTF-8 
> -Dmaven.wagon.http.ssl.easy=false -Dmaven.wagon.rto=1 $LEIN_JVM_OPTS 
> -Dleiningen.original.pwd="$ORIGINAL_PWD" -Dleiningen.script="$0" -classpath 
> "$CLASSPATH" clojure.main -m leiningen.core.main "$@"
>
>
> Looking for some advice about the simplest way to get past this error? 
>
>
>
>
>

-- 
You received this message because you are subscribed to the Google
Groups "Clojure" group.
To post to this group, send email to clojure@googlegroups.com
Note that posts from new members are moderated - please be patient with your 
first post.
To unsubscribe from this group, send email to
clojure+unsubscr...@googlegroups.com
For more options, visit this group at
http://groups.google.com/group/clojure?hl=en
--- 
You received this message because you are subscribed to the Google Groups 
"Clojure" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to clojure+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: Received fatal alert: protocol_version

[lawrence . krubner]

Saw a suggestion here:

https://github.com/technomancy/leiningen/issues/2364

and tried it:

LEIN_JVM_OPTS=-Dhttps.protocols=TLSv1.2 lein -U deps

but got:

Could not transfer artifact io.javalin:javalin:pom:2.4.0 from/to central 
(https://repo1.maven.org/maven2/): java.lang.RuntimeException: Unexpected 
error: java.security.InvalidAlgorithmParameterException: the trustAnchors 
parameter must be non-empty




On Thursday, November 22, 2018 at 3:37:31 PM UTC-5, lawrence...@gmail.com 
wrote:
>
> I've been away from Clojure for a year and I'm just coming back to it. I 
> assume some of my dependencies are out of date. I'm working on a project 
> that is 90% Java with a Clojure wrapper. I tried this:
>
> lein uber
>
> I got a "protocol version" error which seems to have something to do with 
> a new level of security at Maven. 
>
> So I tried this:
>
>  LEIN_JVM_OPTS=-Dhttps.protocols=TLSv1.2   lein uber
>
> and got all this: 
>
> # A fatal error has been detected by the Java Runtime Environment:
> #
> #  SIGBUS (0xa) at pc=106510692, pid=66917, tid=123145404563456
> #
> # JRE version: 7.0-b222
> # Java VM: OpenJDK 64-Bit Server VM (21.0-b17 mixed mode bsd-amd64 
> compressed oops)
> # Problematic frame:
> # J  clojure.lang.Symbol.equals(Ljava/lang/Object;)Z
> #
> # Failed to write core dump. Core dumps have been disabled. To enable core 
> dumping, try "ulimit -c unlimited" before starting Java again
> #
> # An error report file with more information is saved as:
> # 
> /Users/sugar/projects/vegetable/repos/cortex/cortext-clj/hs_err_pid66917.log
> #
> # If you would like to submit a bug report, please visit:
> #   http://bugreport.sun.com/bugreport/crash.jsp
> #
> /usr/local/bin/lein: line 152: 66917 Abort trap: 6  
>  "$LEIN_JAVA_CMD" "${BOOTCLASSPATH[@]}" -Dfile.encoding=UTF-8 
> -Dmaven.wagon.http.ssl.easy=false -Dmaven.wagon.rto=1 $LEIN_JVM_OPTS 
> -Dleiningen.original.pwd="$ORIGINAL_PWD" -Dleiningen.script="$0" -classpath 
> "$CLASSPATH" clojure.main -m leiningen.core.main "$@"
>
>
> Looking for some advice about the simplest way to get past this error? 
>
>
>
>
>

-- 
You received this message because you are subscribed to the Google
Groups "Clojure" group.
To post to this group, send email to clojure@googlegroups.com
Note that posts from new members are moderated - please be patient with your 
first post.
To unsubscribe from this group, send email to
clojure+unsubscr...@googlegroups.com
For more options, visit this group at
http://groups.google.com/group/clojure?hl=en
--- 
You received this message because you are subscribed to the Google Groups 
"Clojure" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to clojure+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Received fatal alert: protocol_version

[lawrence . krubner]

I've been away from Clojure for a year and I'm just coming back to it. I 
assume some of my dependencies are out of date. I'm working on a project 
that is 90% Java with a Clojure wrapper. I tried this:

lein uber

I got a "protocol version" error which seems to have something to do with a 
new level of security at Maven. 

So I tried this:

 LEIN_JVM_OPTS=-Dhttps.protocols=TLSv1.2   lein uber

and got all this: 

# A fatal error has been detected by the Java Runtime Environment:
#
#  SIGBUS (0xa) at pc=106510692, pid=66917, tid=123145404563456
#
# JRE version: 7.0-b222
# Java VM: OpenJDK 64-Bit Server VM (21.0-b17 mixed mode bsd-amd64 
compressed oops)
# Problematic frame:
# J  clojure.lang.Symbol.equals(Ljava/lang/Object;)Z
#
# Failed to write core dump. Core dumps have been disabled. To enable core 
dumping, try "ulimit -c unlimited" before starting Java again
#
# An error report file with more information is saved as:
# 
/Users/sugar/projects/vegetable/repos/cortex/cortext-clj/hs_err_pid66917.log
#
# If you would like to submit a bug report, please visit:
#   http://bugreport.sun.com/bugreport/crash.jsp
#
/usr/local/bin/lein: line 152: 66917 Abort trap: 6  
 "$LEIN_JAVA_CMD" "${BOOTCLASSPATH[@]}" -Dfile.encoding=UTF-8 
-Dmaven.wagon.http.ssl.easy=false -Dmaven.wagon.rto=1 $LEIN_JVM_OPTS 
-Dleiningen.original.pwd="$ORIGINAL_PWD" -Dleiningen.script="$0" -classpath 
"$CLASSPATH" clojure.main -m leiningen.core.main "$@"


Looking for some advice about the simplest way to get past this error? 




-- 
You received this message because you are subscribed to the Google
Groups "Clojure" group.
To post to this group, send email to clojure@googlegroups.com
Note that posts from new members are moderated - please be patient with your 
first post.
To unsubscribe from this group, send email to
clojure+unsubscr...@googlegroups.com
For more options, visit this group at
http://groups.google.com/group/clojure?hl=en
--- 
You received this message because you are subscribed to the Google Groups 
"Clojure" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to clojure+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: Officially support Vert.x

[lawrence . krubner]

James Gatannah,

I apologize for hijacking this thread, but what did you mean here: 

> The one useful thing I could find that Vert.x provides out of the box 
> that clojure doesn't is the pub/sub messaging. That turned our 
> architecture into spaghetti, so I wouldn't call it a win.

Was there something specific about Vert.x that made pub/sub difficult, or 
did you find that pub/sub itself, as a pattern, undermined your 
architecture? I've used pub/sub among small sets of microservices (5 or 6 
services) and it worked well, though I'd be curious to hear from someone 
who tried to use it with hundreds of services. 




On Sunday, December 31, 2017 at 12:47:50 PM UTC-5, James Gatannah wrote:
>
> Disclaimer: I once spent 3-4 weeks studying Vert.x's design philosophy and 
> architecture. So I'm hardly an expert.
>
> On Saturday, December 30, 2017 at 9:31:47 AM UTC-6, Feuer Au wrote:
>>
>> Hmm
>> yes it is almost an absolute requirement since we have already invested a 
>> lot in Java, Kotlin, Javascript and other languages based on Vert.x
>>
>
> I think there's a very important open question here: *what* are you 
> actually using Vert.x for?
>  
>
>> and for Clojure & Haskell are pretty new candidates for us, we need to 
>> find a way to persuade our programmers to use these languages
>>
> and Vert.x is almost the only way to persuade our lovely programmers to 
>> give it a try
>>
>  
>
>> If we dont use Vert.x, those guys may resist to try a new language since 
>> they need to get used to many new softwares e.g. maven -> leiningen
>> some times people may not like changes^_^ 
>>
>
> It's been my experience that, if you have to persuade the programmers to 
> use clojure, they probably aren't going to approach it correctly.
>
> They'll try to write it the same way they'd write java. Then they'll 
> decide that it's awful because the startup time is too slow, the syntax is 
> too weird, and the immutable data structures add a lot of overhead for no 
> real benefit.
>
> Now, if you have someone on the team who's already realized that there are 
> a ton of benefits (like stability in production, much faster development 
> time, fewer bugs, less code to maintain, and general developer happiness 
> all come to mind), then allowing them to show the others the benefits is a 
> great option.
>
> But someone on the team really needs to already  understand that this *is* 
> a better way.
>
>>  
>
>> Now, building on what other people have already written:
>
> If you have a bunch of separate stand-alone microservices that are using 
> vert.x to communicate, there's no reason you couldn't write one in clojure. 
> Just call the Vert.x pieces the way you would any other Java library. I 
> don't know how much effort would be involved in getting it to play nicely 
> with the actual Vert.x ecosystem. It's been a while since I looked at this, 
> but I remember that one of the main selling points was the ability to spin 
> verticles up and down on demand.
>
> That sort of thing really isn't clojure's strength. It's really meant to 
> be part of a process that runs for months or years. You don't have to use 
> it that way, of course, and I bet plenty of others on this list are not. 
> But, if you need something that automatically scales up pretty much 
> instantly when demand increases, then clojure may not be your best bet.
>
> And, if you don't, then why are you using Vert.x?
>
> For me, trying to use Vert.x with clojure was incredibly frustrating. In a 
> lot of ways, it felt like they're trying to solve the same basic problems 
> in two totally different ways. 
>
> One of clojure's main selling points is sane multi-threading. Using that 
> involves fighting the Vert.x ecosystem, which is at least partially based 
> on the premise that multi-threading is sheer evil and should really be 
> avoided. I never really into this, because I barely scratched the surface, 
> and it seemed like a really nasty can of worms to unleash.
>
> As others have said, a huge chunk of what Vert.x provides is already baked 
> into clojure. But the different approaches really aren't compatible. So 
> you'll be fighting either the language or the library on some pretty 
> fundamental decisions.
>
> The one useful thing I could find that Vert.x provides out of the box that 
> clojure doesn't is the pub/sub messaging. That turned our architecture into 
> spaghetti, so I wouldn't call it a win.
>
> Then, yeah, there's the codegen thing which is the part you're really 
> asking about.
>
> Why would anyone use an external codegen tool for a lisp? You already have 
> [arguably] the best codegen tool the world has run across baked into the 
> language, in the form of macros.
>
> If I really had to use Vert.x (and I was working in a polyglot environment 
> where this was an option), I'd just write stand-alone clojure microservices 
> that used Vert.x as a communications library. Then I'd point out how much 
> time/energy is being wasted on the java/kotlin

Re: Russ olsen's Clojure Book

[lawrence . krubner]

Stuart Sierra wrote a bit about Functional Design Patters here:

https://www.infoq.com/presentations/Clojure-Design-Patterns

Michael Bevilacqua-Linn wrote "Functional Programming Patterns in Scala and 
Clojure":

https://www.amazon.com/Functional-Programming-Patterns-Scala-Clojure/dp/1937785475/

And I reiterated all the standard arguments about why Lisps don't typically 
have the classical Design Patterns here:

http://www.smashcompany.com/technology/object-oriented-programming-is-an-expensive-disaster-which-must-end




On Wednesday, June 29, 2011 at 11:09:56 PM UTC-4, Sayth Renshaw wrote:
>
>
> Just wanted to put a shout out to Russ Olsen to see what would be 
> needed to get a Russ Olsen book on clojure to happen. I am reading 
> design principles in Ruby and its a great read, I feel I am learning 
> much moe than just Ruby which is why I am reading it. 
>
> I woould absolutely love to read how Russ would apply these design 
> principles to Clojure a more functional language. His books are all 5 
> star reads(amazon ratings) and would greatly enjoy being able to read 
> a Russ Olsen clojure book. 
>
> Is there any way we could help this to happen? Is anyone else 
> interested in such a book? 
>
> Sayth

-- 
You received this message because you are subscribed to the Google
Groups "Clojure" group.
To post to this group, send email to clojure@googlegroups.com
Note that posts from new members are moderated - please be patient with your 
first post.
To unsubscribe from this group, send email to
clojure+unsubscr...@googlegroups.com
For more options, visit this group at
http://groups.google.com/group/clojure?hl=en
--- 
You received this message because you are subscribed to the Google Groups 
"Clojure" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to clojure+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Am I being stupid regarding how much memory I use?

[lawrence . krubner]

I recently worked on a minor project that nevertheless needed to use 10 
gigs of RAM. It ran on a reasonably powerful server, yet it taxed that 
server. And I wondered, how are people scaling up such processes? If my 
approach was naive, what does the less naive approach look like? 

I wrote a simple app that pulled data from a MySQL database, denormalized 
it, and then stored it in ElasticSearch. It pulled about 4 million 
documents from MySQL. Parts of the data needed to be built up into complex 
structures (maps, vectors) before being put into ElasticSearch. In the end, 
the 4 million rows from MySQL became 1.5 million documents in ElasticSearch.

I was wondering, what if, instead of 4 million documents, I needed to 
process 400 million documents? I assume I would have to distribute the work 
over several machines? I'm curious what are some of the most common routes 
for doing so? Would this be the situation where people would start to use 
something like Onyx or Storm or Hadoop? I looked at Spark but it seems to 
be for a different use case, more about querying that denormalizing. 
Likewise, dumping everything onto S3 and then using something like Athena 
seems to be more for querying than denormalizing. 

For unrelated reasons, I am moving toward the architecture where all data 
is stored in Kafka. I suppose I could write a denormalizing app that reads 
over Kafka and builds up the data and then inserts it to ElasticSearch, 
though I suppose, on the narrow issue of memory usage, using Kafka is no 
different than using using MySQL.

So, I'm asking about common patterns here. When folks have an app that 
needs more RAM than a typical server, what is the first and most common 
steps they take? 







-- 
You received this message because you are subscribed to the Google
Groups "Clojure" group.
To post to this group, send email to clojure@googlegroups.com
Note that posts from new members are moderated - please be patient with your 
first post.
To unsubscribe from this group, send email to
clojure+unsubscr...@googlegroups.com
For more options, visit this group at
http://groups.google.com/group/clojure?hl=en
--- 
You received this message because you are subscribed to the Google Groups 
"Clojure" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to clojure+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: Doc strings for complex cases?

[lawrence . krubner]

I'm curious if it would be enough to write a macro that replaces defn in 
your code? The macro would secretly generate a different function for every 
arity, and therefore it could have a different doc-string for every arity, 
but it wouldn't have to appear that way in your code. Is that enough, or do 
you need something more fundamental? If so, can you say why? 

Seems like you could almost certainly do this for defmulti?


On Sunday, November 5, 2017 at 7:59:02 AM UTC-5, Tim wrote:
>
> I'm seeing a need to assign a doc string per arity case for defn and also 
> for each defmethod as opposed to storing it all in defmulti. Any solutions 
> or plans for this? 
> Tim

-- 
You received this message because you are subscribed to the Google
Groups "Clojure" group.
To post to this group, send email to clojure@googlegroups.com
Note that posts from new members are moderated - please be patient with your 
first post.
To unsubscribe from this group, send email to
clojure+unsubscr...@googlegroups.com
For more options, visit this group at
http://groups.google.com/group/clojure?hl=en
--- 
You received this message because you are subscribed to the Google Groups 
"Clojure" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to clojure+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Feed predicates from clojure.spec to the inference engine/type checker from the Shen java port?

[lawrence . krubner]

This sounds like an interesting idea. Does anyone know if this could work? 

https://news.ycombinator.com/item?id=15569058

The allure of Shen is that it offers a type system that is, roughly 
speaking, very similar to what you get with clojure.spec. You construct 
"predicates", what Shen calls sequents, which hold about your data. Shen 
goes a step further than clojure.spec in that it offers a way to prove that 
those predicates hold locally to each function and globally through your 
program.

I wonder if it might be possible to feed predicates from clojure.spec to 
the inference engine/type checker from the Shen java port to gain static 
type checking. I wish I had more time to work on this kind of fundamental 
research, but alas I'm merely a working programmer schlepping data to and 
fro.

-- 
You received this message because you are subscribed to the Google
Groups "Clojure" group.
To post to this group, send email to clojure@googlegroups.com
Note that posts from new members are moderated - please be patient with your 
first post.
To unsubscribe from this group, send email to
clojure+unsubscr...@googlegroups.com
For more options, visit this group at
http://groups.google.com/group/clojure?hl=en
--- 
You received this message because you are subscribed to the Google Groups 
"Clojure" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to clojure+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: Nice screencast overview of Specter

[lawrence . krubner]

I surprised that I had never heard of this, despite following the other 
work that Nathan Marz has done. 


On Wednesday, October 18, 2017 at 11:01:24 AM UTC-4, Alan Thompson wrote:
>
> I just saw this on YouTube and thought you may enjoy it:  
> https://youtu.be/rh5J4vacG98
>
> While there is lots of written documentation on Specter, the live-coding 
> and narration of what one is seeing provides some nice examples.
>
> Enjoy,
> Alan
>

-- 
You received this message because you are subscribed to the Google
Groups "Clojure" group.
To post to this group, send email to clojure@googlegroups.com
Note that posts from new members are moderated - please be patient with your 
first post.
To unsubscribe from this group, send email to
clojure+unsubscr...@googlegroups.com
For more options, visit this group at
http://groups.google.com/group/clojure?hl=en
--- 
You received this message because you are subscribed to the Google Groups 
"Clojure" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to clojure+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: (resolve (symbol f)) works at the REPL but not in an uberjar

[lawrence . krubner]

Is there a way I can avoid hard-coding " "denormalize.pull-from-mysql"" ? I 
guess it doesn't matter, but it does seem a little inelegant.


On Thursday, October 12, 2017 at 9:54:36 PM UTC-4, Justin Smith wrote:
>
> you can use (symbol "denormalize.pull-from-mysql" "f") instead
>
> On Thu, Oct 12, 2017 at 6:34 PM > 
> wrote:
>
>> Nevermind. I found that this works, though I think it is ugly and 
>> inelegant: 
>>
>>  resolved-f (resolve (symbol (str "denormalize.pull-from-mysql/" 
>> f)))
>>
>>
>> On Thursday, October 12, 2017 at 9:27:06 PM UTC-4, lawrence...@gmail.com 
>> wrote:
>>>
>>> At the REPL, this works perfectly:
>>>
>>>  > (defn get-users [] [:susan :kwan])
>>>  > 
>>>  > (defn what-is-this-function [f] ((resolve (symbol f
>>>  
>>>  > (what-is-this-function "get-users")
>>>  >[:susan :kwan]
>>>  
>>> In an uberjar this does not work. I read somewhere that the runtime 
>>> name-space is different than the REPL namespace and different from the 
>>> namespace that exists while the compiler compiles the code. 
>>>
>>> So how do I resolve the symbol correctly? 
>>>
>>> I have functions that call a MySQL database, and each function is named 
>>> after a database table. Most of the mechanics of the call are the same, the 
>>> only thing different is the SQL. So I have functions such as "company" and 
>>> "user" and "product". I also have a function called (select). I call select 
>>> with one of the other functions, given as a string: 
>>>
>>> (select "product")
>>>
>>> Inside of (select) I need to turn "product" into a resolved symbol that 
>>> I can call. 
>>>
>>> Afterwards, I store the results in a map in an atom. I want to use the 
>>> database names as the keys. So "product" should be the key that holds the 
>>> results of the function "product". 
>>>
>>> But what I need to do to correctly resolve the string to a function? 
>>>
>>>
>>>
>>>
>>>
>>> -- 
>> You received this message because you are subscribed to the Google
>> Groups "Clojure" group.
>> To post to this group, send email to clo...@googlegroups.com 
>> 
>> Note that posts from new members are moderated - please be patient with 
>> your first post.
>> To unsubscribe from this group, send email to
>> clojure+u...@googlegroups.com 
>> For more options, visit this group at
>> http://groups.google.com/group/clojure?hl=en
>> --- 
>> You received this message because you are subscribed to the Google Groups 
>> "Clojure" group.
>> To unsubscribe from this group and stop receiving emails from it, send an 
>> email to clojure+u...@googlegroups.com .
>> For more options, visit https://groups.google.com/d/optout.
>>
>

-- 
You received this message because you are subscribed to the Google
Groups "Clojure" group.
To post to this group, send email to clojure@googlegroups.com
Note that posts from new members are moderated - please be patient with your 
first post.
To unsubscribe from this group, send email to
clojure+unsubscr...@googlegroups.com
For more options, visit this group at
http://groups.google.com/group/clojure?hl=en
--- 
You received this message because you are subscribed to the Google Groups 
"Clojure" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to clojure+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: possibly a Clojure question or possibly an AWS question: slow writes to durable-queue

[lawrence . krubner]

Following Daniel Compton's suggestion, I turned on logging for GC. I don't 
see it happening more often, but the slow down does seem related to the 
moment when the app hits the maximum memory allowed. It had been running 
with 4G, so I increased that to 7G, so it goes longer now before it hits 
98% memory usage, but it does hit it eventually and then everything crawls 
to a very slow speed. Not sure how much memory I would have to use to avoid 
using up almost all of the memory. I suppose I'll figure that out via trial 
and error. Until I can figure that out, nearly all other performance tricks 
seems a bit besides the point. 



On Thursday, October 12, 2017 at 9:01:23 PM UTC-4, Nathan Fisher wrote:
>
> Hi!
>
> Can you change one of the variables? Specifically can you replicate this 
> on your local machine? If it happens locally then I would focus on 
> something in the JVM eco-system.
>
> If you can't replicate it locally then it's possibly AWS specific. It 
> sounds like you're using a t2.large or m4.xlarge. If it's the prior you may 
> very well be contending between with your network bandwidth. EC2's host 
> drive (EBS) is a networked drive which is split between your standard 
> network traffic and the drive volume. If that's the issue then you might 
> need to look at provisioned IOPs. A quick(ish) way to test that hypothesis 
> is to provision a host with high networking performance and provisioned 
> IOPs.
>
> Cheers,
> Nathan
>
> On Fri, 13 Oct 2017 at 00:05 > wrote:
>
>> Daniel Compton, good suggestion. I've increased the memory to see if I 
>> can postpone the GCs, and I'll log that more carefully. 
>>
>>
>> On Wednesday, October 11, 2017 at 8:35:44 PM UTC-4, Daniel Compton wrote:
>>
>>> Without more information it's hard to tell, but this looks a like it 
>>> could be a garbage collection issue. Can you run your test again and add 
>>> some logging/monitoring to show each garbage collection? If my hunch is 
>>> right, you'll see garbage collections getting more and more frequent until 
>>> they take up nearly all the CPU time, preventing much forward progress 
>>> writing to the queue.
>>>
>>> If it's AWS based throttling, then CloudWatch monitoring 
>>> http://docs.aws.amazon.com/AWSEC2/latest/UserGuide/monitoring-volume-status.html#using_cloudwatch_ebs
>>>  might 
>>> show you some hints. You could also test with an NVMe drive attached, just 
>>> to see if disk bandwidth is the issue.
>>>
>>> On Thu, Oct 12, 2017 at 11:58 AM Justin Smith  
>>> wrote:
>>>
>> a small thing here, if memory usage is important you should be building 
 and running an uberjar instead of using lein on the server (this also has 
 other benefits), and if you are doing that your project.clj jvm-opts are 
 not used, you have to configure your java command line in aws instead

 On Wed, Oct 11, 2017 at 3:52 PM  wrote:

>>> I can't figure out if this is a Clojure question or an AWS question. And 
> if it is a Clojure question, I can't figure out if it is more of a 
> general 
> JVM question, or if it is specific to some library such as durable-queue. 
> I 
> can redirect my question elsewhere, if people think this is an AWS 
> question. 
>
> In my project.clj, I try to give my app a lot of memory:
>
>   :jvm-opts ["-Xms7g" "-Xmx7g" "-XX:-UseCompressedOops"])
>
> And the app starts off pulling data from MySQL and writing it to 
> Durable-Queue at a rapid rate. ( 
> https://github.com/Factual/durable-queue )
>
> I have some logging set up to report every 30 seconds.
>
> :enqueued 370137,
>
> 30 seconds later:
>
> :enqueued 608967,
>
> 30 seconds later: 
>
> :enqueued 828950,
>
> It's a dramatic slowdown. The app is initially writing to the queue at 
> faster than 10,000 documents a second, but it slows steadily, and after 
> 10 
> minutes it writes less than 1,000 documents per second. Since I have to 
> write a few million documents, 10,000 a second is the slowest speed I can 
> live with. 
>
> The queues are in the /tmp folder of an AWS instance that has plenty 
> of disk space, 4 CPUs, and 16 gigs of RAM. 
>
> Why does the app slow down so much? I had 4 thoughts:
>
> 1.) the app struggles as it hits a memory limit
>
> 2.) memory bandwidth is the problem
>
> 3.) AWS is enforcing some weird IOPS limit
>
> 4.) durable-queue is misbehaving
>
> As to possibility #1, I notice the app starts like this:
>
> Memory in use (percentage/used/max-heap): (\"66%\" \"2373M\" \"3568M\")
>
> but 60 seconds later I see: 
>
> Memory in use (percentage/used/max-heap): (\"94%\" \"3613M\" \"3819M\")
>
> So I've run out of allowed memory. But why is that? I thought I gave 
> this app 7 gigs: 
>
>   :jvm-opts ["-Xms7g" "-Xmx7g" "-XX:-UseCompressedOops"])
>
> As to possibility #2, I

Re: (resolve (symbol f)) works at the REPL but not in an uberjar

[lawrence . krubner]

Nevermind. I found that this works, though I think it is ugly and 
inelegant: 

 resolved-f (resolve (symbol (str "denormalize.pull-from-mysql/" 
f)))


On Thursday, October 12, 2017 at 9:27:06 PM UTC-4, lawrence...@gmail.com 
wrote:
>
> At the REPL, this works perfectly:
>
>  > (defn get-users [] [:susan :kwan])
>  > 
>  > (defn what-is-this-function [f] ((resolve (symbol f
>  
>  > (what-is-this-function "get-users")
>  >[:susan :kwan]
>  
> In an uberjar this does not work. I read somewhere that the runtime 
> name-space is different than the REPL namespace and different from the 
> namespace that exists while the compiler compiles the code. 
>
> So how do I resolve the symbol correctly? 
>
> I have functions that call a MySQL database, and each function is named 
> after a database table. Most of the mechanics of the call are the same, the 
> only thing different is the SQL. So I have functions such as "company" and 
> "user" and "product". I also have a function called (select). I call select 
> with one of the other functions, given as a string: 
>
> (select "product")
>
> Inside of (select) I need to turn "product" into a resolved symbol that I 
> can call. 
>
> Afterwards, I store the results in a map in an atom. I want to use the 
> database names as the keys. So "product" should be the key that holds the 
> results of the function "product". 
>
> But what I need to do to correctly resolve the string to a function? 
>
>
>
>
>
>

-- 
You received this message because you are subscribed to the Google
Groups "Clojure" group.
To post to this group, send email to clojure@googlegroups.com
Note that posts from new members are moderated - please be patient with your 
first post.
To unsubscribe from this group, send email to
clojure+unsubscr...@googlegroups.com
For more options, visit this group at
http://groups.google.com/group/clojure?hl=en
--- 
You received this message because you are subscribed to the Google Groups 
"Clojure" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to clojure+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

(resolve (symbol f)) works at the REPL but not in an uberjar

[lawrence . krubner]

At the REPL, this works perfectly:

 > (defn get-users [] [:susan :kwan])
 > 
 > (defn what-is-this-function [f] ((resolve (symbol f
 
 > (what-is-this-function "get-users")
 >[:susan :kwan]
 
In an uberjar this does not work. I read somewhere that the runtime 
name-space is different than the REPL namespace and different from the 
namespace that exists while the compiler compiles the code. 

So how do I resolve the symbol correctly? 

I have functions that call a MySQL database, and each function is named 
after a database table. Most of the mechanics of the call are the same, the 
only thing different is the SQL. So I have functions such as "company" and 
"user" and "product". I also have a function called (select). I call select 
with one of the other functions, given as a string: 

(select "product")

Inside of (select) I need to turn "product" into a resolved symbol that I 
can call. 

Afterwards, I store the results in a map in an atom. I want to use the 
database names as the keys. So "product" should be the key that holds the 
results of the function "product". 

But what I need to do to correctly resolve the string to a function? 





-- 
You received this message because you are subscribed to the Google
Groups "Clojure" group.
To post to this group, send email to clojure@googlegroups.com
Note that posts from new members are moderated - please be patient with your 
first post.
To unsubscribe from this group, send email to
clojure+unsubscr...@googlegroups.com
For more options, visit this group at
http://groups.google.com/group/clojure?hl=en
--- 
You received this message because you are subscribed to the Google Groups 
"Clojure" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to clojure+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: possibly a Clojure question or possibly an AWS question: slow writes to durable-queue

[lawrence . krubner]

Daniel Compton, good suggestion. I've increased the memory to see if I can 
postpone the GCs, and I'll log that more carefully. 


On Wednesday, October 11, 2017 at 8:35:44 PM UTC-4, Daniel Compton wrote:
>
> Without more information it's hard to tell, but this looks a like it could 
> be a garbage collection issue. Can you run your test again and add some 
> logging/monitoring to show each garbage collection? If my hunch is right, 
> you'll see garbage collections getting more and more frequent until they 
> take up nearly all the CPU time, preventing much forward progress writing 
> to the queue.
>
> If it's AWS based throttling, then CloudWatch monitoring 
> http://docs.aws.amazon.com/AWSEC2/latest/UserGuide/monitoring-volume-status.html#using_cloudwatch_ebs
>  might 
> show you some hints. You could also test with an NVMe drive attached, just 
> to see if disk bandwidth is the issue.
>
> On Thu, Oct 12, 2017 at 11:58 AM Justin Smith  > wrote:
>
>> a small thing here, if memory usage is important you should be building 
>> and running an uberjar instead of using lein on the server (this also has 
>> other benefits), and if you are doing that your project.clj jvm-opts are 
>> not used, you have to configure your java command line in aws instead
>>
>> On Wed, Oct 11, 2017 at 3:52 PM > 
>> wrote:
>>
>>> I can't figure out if this is a Clojure question or an AWS question. And 
>>> if it is a Clojure question, I can't figure out if it is more of a general 
>>> JVM question, or if it is specific to some library such as durable-queue. I 
>>> can redirect my question elsewhere, if people think this is an AWS 
>>> question. 
>>>
>>> In my project.clj, I try to give my app a lot of memory:
>>>
>>>   :jvm-opts ["-Xms7g" "-Xmx7g" "-XX:-UseCompressedOops"])
>>>
>>> And the app starts off pulling data from MySQL and writing it to 
>>> Durable-Queue at a rapid rate. ( 
>>> https://github.com/Factual/durable-queue )
>>>
>>> I have some logging set up to report every 30 seconds.
>>>
>>> :enqueued 370137,
>>>
>>> 30 seconds later:
>>>
>>> :enqueued 608967,
>>>
>>> 30 seconds later: 
>>>
>>> :enqueued 828950,
>>>
>>> It's a dramatic slowdown. The app is initially writing to the queue at 
>>> faster than 10,000 documents a second, but it slows steadily, and after 10 
>>> minutes it writes less than 1,000 documents per second. Since I have to 
>>> write a few million documents, 10,000 a second is the slowest speed I can 
>>> live with. 
>>>
>>> The queues are in the /tmp folder of an AWS instance that has plenty of 
>>> disk space, 4 CPUs, and 16 gigs of RAM. 
>>>
>>> Why does the app slow down so much? I had 4 thoughts:
>>>
>>> 1.) the app struggles as it hits a memory limit
>>>
>>> 2.) memory bandwidth is the problem
>>>
>>> 3.) AWS is enforcing some weird IOPS limit
>>>
>>> 4.) durable-queue is misbehaving
>>>
>>> As to possibility #1, I notice the app starts like this:
>>>
>>> Memory in use (percentage/used/max-heap): (\"66%\" \"2373M\" \"3568M\")
>>>
>>> but 60 seconds later I see: 
>>>
>>> Memory in use (percentage/used/max-heap): (\"94%\" \"3613M\" \"3819M\")
>>>
>>> So I've run out of allowed memory. But why is that? I thought I gave 
>>> this app 7 gigs: 
>>>
>>>   :jvm-opts ["-Xms7g" "-Xmx7g" "-XX:-UseCompressedOops"])
>>>
>>> As to possibility #2, I found this old post on the Clojure mailist:
>>>
>>> Andy Fingerhut wrote, "one thing I've found in the past on a 2-core 
>>> machine that was achieving much less than 2x speedup was memory bandwidth 
>>> being the limiting factor."
>>>
>>>
>>> https://groups.google.com/forum/#!searchin/clojure/xmx$20xms$20maximum%7Csort:relevance/clojure/48W2eff3caU/HS6u547gtrAJ
>>>
>>> But I am not sure how to test this. 
>>>
>>> As to possibility #3, I'm not sure how AWS enforces its IOPS limits. If 
>>> people think this is the most likely possibility, then I will repost my 
>>> question in an AWS forum. 
>>>
>>> As to possibility #4, durable-queue is well-tested and used in a lot of 
>>> projects, and Zach Tellman is smart and makes few mistakes, so I'm doubtful 
>>> that it is to blame, but I do notice that it starts off with 4 active slabs 
>>> and then after 120 seconds, it is only using 1 slab. Is that expected? If 
>>> people think this is the possible problem then I'll ask somewhere specific 
>>> to durable-queue
>>>
>>> Overall, my log information looks like this: 
>>>
>>> ("\nStats about from-mysql-to-tables-queue: " {"message" {:num-slabs 
>>> 3, :num-active-slabs 2, :enqueued 370137, :retried 0, :completed 369934, 
>>> :in-progress 10}})
>>>
>>> ("\nResource usage: " "Memory in use (percentage/used/max-heap): 
>>> (\"66%\" \"2373M\" \"3568M\")\n\nCPU usage (how-many-cpu's/load-average): 
>>>  [4 5.05]\n\nFree memory in jvm: [1171310752]")
>>>
>>> 30 seconds later
>>>
>>> ("\nStats about from-mysql-to-tables-queue: " {"message" {:num-slabs 
>>> 4, :num-active-slabs 4, :enqueued 608967, :retried 0, :completed 608511, 
>>> :in-progress 10}})
>>> 
>>> ("\

Re: possibly a Clojure question or possibly an AWS question: slow writes to durable-queue

[lawrence . krubner]

Justin Smith, thanks, I've created an Uberjar that I now run under 
Supervisord on an EC2 instance, and I set the JVM options via the command 
that Supervisord calls. 


On Wednesday, October 11, 2017 at 6:58:52 PM UTC-4, Justin Smith wrote:
>
> a small thing here, if memory usage is important you should be building 
> and running an uberjar instead of using lein on the server (this also has 
> other benefits), and if you are doing that your project.clj jvm-opts are 
> not used, you have to configure your java command line in aws instead
>
> On Wed, Oct 11, 2017 at 3:52 PM > 
> wrote:
>
>> I can't figure out if this is a Clojure question or an AWS question. And 
>> if it is a Clojure question, I can't figure out if it is more of a general 
>> JVM question, or if it is specific to some library such as durable-queue. I 
>> can redirect my question elsewhere, if people think this is an AWS 
>> question. 
>>
>> In my project.clj, I try to give my app a lot of memory:
>>
>>   :jvm-opts ["-Xms7g" "-Xmx7g" "-XX:-UseCompressedOops"])
>>
>> And the app starts off pulling data from MySQL and writing it to 
>> Durable-Queue at a rapid rate. ( https://github.com/Factual/durable-queue 
>> )
>>
>> I have some logging set up to report every 30 seconds.
>>
>> :enqueued 370137,
>>
>> 30 seconds later:
>>
>> :enqueued 608967,
>>
>> 30 seconds later: 
>>
>> :enqueued 828950,
>>
>> It's a dramatic slowdown. The app is initially writing to the queue at 
>> faster than 10,000 documents a second, but it slows steadily, and after 10 
>> minutes it writes less than 1,000 documents per second. Since I have to 
>> write a few million documents, 10,000 a second is the slowest speed I can 
>> live with. 
>>
>> The queues are in the /tmp folder of an AWS instance that has plenty of 
>> disk space, 4 CPUs, and 16 gigs of RAM. 
>>
>> Why does the app slow down so much? I had 4 thoughts:
>>
>> 1.) the app struggles as it hits a memory limit
>>
>> 2.) memory bandwidth is the problem
>>
>> 3.) AWS is enforcing some weird IOPS limit
>>
>> 4.) durable-queue is misbehaving
>>
>> As to possibility #1, I notice the app starts like this:
>>
>> Memory in use (percentage/used/max-heap): (\"66%\" \"2373M\" \"3568M\")
>>
>> but 60 seconds later I see: 
>>
>> Memory in use (percentage/used/max-heap): (\"94%\" \"3613M\" \"3819M\")
>>
>> So I've run out of allowed memory. But why is that? I thought I gave this 
>> app 7 gigs: 
>>
>>   :jvm-opts ["-Xms7g" "-Xmx7g" "-XX:-UseCompressedOops"])
>>
>> As to possibility #2, I found this old post on the Clojure mailist:
>>
>> Andy Fingerhut wrote, "one thing I've found in the past on a 2-core 
>> machine that was achieving much less than 2x speedup was memory bandwidth 
>> being the limiting factor."
>>
>>
>> https://groups.google.com/forum/#!searchin/clojure/xmx$20xms$20maximum%7Csort:relevance/clojure/48W2eff3caU/HS6u547gtrAJ
>>
>> But I am not sure how to test this. 
>>
>> As to possibility #3, I'm not sure how AWS enforces its IOPS limits. If 
>> people think this is the most likely possibility, then I will repost my 
>> question in an AWS forum. 
>>
>> As to possibility #4, durable-queue is well-tested and used in a lot of 
>> projects, and Zach Tellman is smart and makes few mistakes, so I'm doubtful 
>> that it is to blame, but I do notice that it starts off with 4 active slabs 
>> and then after 120 seconds, it is only using 1 slab. Is that expected? If 
>> people think this is the possible problem then I'll ask somewhere specific 
>> to durable-queue
>>
>> Overall, my log information looks like this: 
>>
>> ("\nStats about from-mysql-to-tables-queue: " {"message" {:num-slabs 
>> 3, :num-active-slabs 2, :enqueued 370137, :retried 0, :completed 369934, 
>> :in-progress 10}})
>>
>> ("\nResource usage: " "Memory in use (percentage/used/max-heap): 
>> (\"66%\" \"2373M\" \"3568M\")\n\nCPU usage (how-many-cpu's/load-average): 
>>  [4 5.05]\n\nFree memory in jvm: [1171310752]")
>>
>> 30 seconds later
>>
>> ("\nStats about from-mysql-to-tables-queue: " {"message" {:num-slabs 
>> 4, :num-active-slabs 4, :enqueued 608967, :retried 0, :completed 608511, 
>> :in-progress 10}})
>> 
>> ("\nResource usage: " "Memory in use (percentage/used/max-heap): 
>> (\"76%\" \"2752M\" \"3611M\")\n\nCPU usage (how-many-cpu's/load-average): 
>>  [4 5.87]\n\nFree memory in jvm: [901122456]")
>>
>> 30 seconds later
>> 
>> ("\nStats about from-mysql-to-tables-queue: " {"message" {:num-slabs 
>> 4, :num-active-slabs 3, :enqueued 828950, :retried 0, :completed 828470, 
>> :in-progress 10}})
>> 
>> ("\nResource usage: " "Memory in use (percentage/used/max-heap): 
>> (\"94%\" \"3613M\" \"3819M\")\n\nCPU usage (how-many-cpu's/load-average): 
>>  [4 6.5]\n\nFree memory in jvm: [216459664]")
>> 
>> 30 seconds later
>>
>> ("\nStats about from-mysql-to-tables-queue: " {"message" {:num-slabs 
>> 1, :num-active-slabs 1, :enqueued 1051974, :retried 0, :completed 1051974, 
>> :in-progress 0}})
>>
>>
>> -

possibly a Clojure question or possibly an AWS question: slow writes to durable-queue

[lawrence . krubner]

I can't figure out if this is a Clojure question or an AWS question. And if 
it is a Clojure question, I can't figure out if it is more of a general JVM 
question, or if it is specific to some library such as durable-queue. I can 
redirect my question elsewhere, if people think this is an AWS question. 

In my project.clj, I try to give my app a lot of memory:

  :jvm-opts ["-Xms7g" "-Xmx7g" "-XX:-UseCompressedOops"])

And the app starts off pulling data from MySQL and writing it to 
Durable-Queue at a rapid rate. ( https://github.com/Factual/durable-queue )

I have some logging set up to report every 30 seconds.

:enqueued 370137,

30 seconds later:

:enqueued 608967,

30 seconds later: 

:enqueued 828950,

It's a dramatic slowdown. The app is initially writing to the queue at 
faster than 10,000 documents a second, but it slows steadily, and after 10 
minutes it writes less than 1,000 documents per second. Since I have to 
write a few million documents, 10,000 a second is the slowest speed I can 
live with. 

The queues are in the /tmp folder of an AWS instance that has plenty of 
disk space, 4 CPUs, and 16 gigs of RAM. 

Why does the app slow down so much? I had 4 thoughts:

1.) the app struggles as it hits a memory limit

2.) memory bandwidth is the problem

3.) AWS is enforcing some weird IOPS limit

4.) durable-queue is misbehaving

As to possibility #1, I notice the app starts like this:

Memory in use (percentage/used/max-heap): (\"66%\" \"2373M\" \"3568M\")

but 60 seconds later I see: 

Memory in use (percentage/used/max-heap): (\"94%\" \"3613M\" \"3819M\")

So I've run out of allowed memory. But why is that? I thought I gave this 
app 7 gigs: 

  :jvm-opts ["-Xms7g" "-Xmx7g" "-XX:-UseCompressedOops"])

As to possibility #2, I found this old post on the Clojure mailist:

Andy Fingerhut wrote, "one thing I've found in the past on a 2-core machine 
that was achieving much less than 2x speedup was memory bandwidth being the 
limiting factor."

https://groups.google.com/forum/#!searchin/clojure/xmx$20xms$20maximum%7Csort:relevance/clojure/48W2eff3caU/HS6u547gtrAJ

But I am not sure how to test this. 

As to possibility #3, I'm not sure how AWS enforces its IOPS limits. If 
people think this is the most likely possibility, then I will repost my 
question in an AWS forum. 

As to possibility #4, durable-queue is well-tested and used in a lot of 
projects, and Zach Tellman is smart and makes few mistakes, so I'm doubtful 
that it is to blame, but I do notice that it starts off with 4 active slabs 
and then after 120 seconds, it is only using 1 slab. Is that expected? If 
people think this is the possible problem then I'll ask somewhere specific 
to durable-queue

Overall, my log information looks like this: 

("\nStats about from-mysql-to-tables-queue: " {"message" {:num-slabs 3, 
:num-active-slabs 2, :enqueued 370137, :retried 0, :completed 369934, 
:in-progress 10}})

("\nResource usage: " "Memory in use (percentage/used/max-heap): 
(\"66%\" \"2373M\" \"3568M\")\n\nCPU usage (how-many-cpu's/load-average): 
 [4 5.05]\n\nFree memory in jvm: [1171310752]")

30 seconds later

("\nStats about from-mysql-to-tables-queue: " {"message" {:num-slabs 4, 
:num-active-slabs 4, :enqueued 608967, :retried 0, :completed 608511, 
:in-progress 10}})

("\nResource usage: " "Memory in use (percentage/used/max-heap): 
(\"76%\" \"2752M\" \"3611M\")\n\nCPU usage (how-many-cpu's/load-average): 
 [4 5.87]\n\nFree memory in jvm: [901122456]")

30 seconds later

("\nStats about from-mysql-to-tables-queue: " {"message" {:num-slabs 4, 
:num-active-slabs 3, :enqueued 828950, :retried 0, :completed 828470, 
:in-progress 10}})

("\nResource usage: " "Memory in use (percentage/used/max-heap): 
(\"94%\" \"3613M\" \"3819M\")\n\nCPU usage (how-many-cpu's/load-average): 
 [4 6.5]\n\nFree memory in jvm: [216459664]")

30 seconds later

("\nStats about from-mysql-to-tables-queue: " {"message" {:num-slabs 1, 
:num-active-slabs 1, :enqueued 1051974, :retried 0, :completed 1051974, 
:in-progress 0}})


-- 
You received this message because you are subscribed to the Google
Groups "Clojure" group.
To post to this group, send email to clojure@googlegroups.com
Note that posts from new members are moderated - please be patient with your 
first post.
To unsubscribe from this group, send email to
clojure+unsubscr...@googlegroups.com
For more options, visit this group at
http://groups.google.com/group/clojure?hl=en
--- 
You received this message because you are subscribed to the Google Groups 
"Clojure" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to clojure+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: Does (into) ever combine 2 numbers into 1?

[lawrence . krubner]

Sorry, this is the problem with working all night. My sleep deprived eyes 
skipped over this line: 

  old-field-value (get old-documents field-name [])

I fetch the value from the wrong map. Such a basic error. 


On Wednesday, October 11, 2017 at 1:44:13 PM UTC-4, lawrence...@gmail.com 
wrote:
>
> I should remove the parens before they confuse anyone. They are added by 
> the log statement. Without the parens:
>
>
>
> {:company_profile_id ["2"], :topic :company, :url ["mikeshawauto.com"]}
>
> field-name: 
> :company_profile_id
>
> field-value:
> ["2"]
>
> but I in the final document I only get: 
>
> "final document: " {:company_profile_id ["2"], :topic :company, :url ["
> mikeshawauto.com"]}
>
> I want: 
>
> "final document: " {:company_profile_id ["2" "2"], :topic :company, :url ["
> mikeshawauto.com"]}
>
>
>
> On Wednesday, October 11, 2017 at 1:42:12 PM UTC-4, lawrence...@gmail.com 
> wrote:
>>
>> So I tried this:
>>
>>   new-field-value (into [] (concat old-field-value 
>> field-value))
>>
>> And I did not get what I expected. Maybe I am sleep deprived, but I don't 
>> see why I can't build up a vector with several values. This is in a map in 
>> an atom. This is the code: 
>>
>>   (swap! documents
>>  (fn [old-documents]
>> (let [
>>   document (get old-documents denormalized-id {})
>>
>>   _ (errors/log document)
>>   _ (errors/log field-name)
>>   _ (errors/log field-value)
>>
>>   old-field-value (get old-documents field-name [])
>>   new-field-value (into [] (concat old-field-value 
>> field-value))
>>   document (assoc document field-name new-field-value)
>>
>>   _ (errors/log "final document: " document)
>>   new-documents (assoc old-documents denormalized-id 
>> document)
>>   ]  
>>   new-documents)
>>
>> Those log statements show me this for old-field-value:
>>
>> ({:company_profile_id ["2"], :topic :company, :url ["mikeshawauto.com"]})
>>
>> field-name: 
>> (:company_profile_id)
>>
>> field-value:
>> (["2"])
>>
>> but I in the final document I only get: 
>>
>> ("final document: " {:company_profile_id ["2"], :topic :company, :url ["
>> mikeshawauto.com"]})
>>
>> I want: 
>>
>> ("final document: " {:company_profile_id ["2", "2"], :topic :company, 
>> :url ["mikeshawauto.com"]})
>>
>> What am I missing here? 
>>
>>
>>
>>
>> On Wednesday, October 11, 2017 at 12:28:59 PM UTC-4, 
>> lawrence...@gmail.com wrote:
>>>
>>> So I have this: 
>>>
>>> ({:company_profile_id ["2"], :topic :company, :url ["mikeshawauto.com
>>> "]})
>>>
>>> And then I get this field name and value: 
>>>
>>> (:company_profile_id)
>>>
>>> (["2"])
>>>
>>> The next 3 lines of code are: 
>>>
>>>   old-field-value (get old-documents field-name [])
>>>   new-field-value (into old-field-value field-value)
>>>   document (assoc document field-name new-field-value)
>>>
>>> And when the function is next called, I end up with :
>>>
>>> ({:company_profile_id ["2"], :topic :company, :url ["mikeshawauto.com
>>> "]})
>>>
>>> I need: 
>>>
>>> ({:company_profile_id ["2", "2"], :topic :company, :url ["
>>> mikeshawauto.com"]})
>>>
>>> So perhaps I've misunderstood what (into) is for? How do I add another 
>>> the values of these 2 vectors together? 
>>>
>>>
>>>
>>>
>>>
>>> On Wednesday, October 11, 2017 at 12:20:55 PM UTC-4, 
>>> lawrence...@gmail.com wrote:

 Using conj instead of into, for no particular reason, except debugging. 
 The document is slowly built-up: 

 ({:company_profile_id [["2"]], :topic :company, :url [["
 mikeshawauto.com"]]})

 ({:company_profile_id [["2"]], :topic :company, :url [["
 mikeshawauto.com"]]})

 ({:company_profile_id [["2"]], :topic :company, :url [["
 mikeshawauto.com"]], :reference_id [["331089191"]]})

 In the transition from row 1 to row 2, I assume the field being added 
 is company_profile_id. Why is there no change? Why isn't a new value added 
 to the vector? I must be missing something obvious. 







 On Wednesday, October 11, 2017 at 11:44:40 AM UTC-4, 
 lawrence...@gmail.com wrote:
>
> I seem unable to figure out where I made a mistake, though this should 
> be simple.
>
> I have two SQL calls that bring back 5 fields:
>
>  SELECT company_profile_id ,  reference_id,  reference_source   FROM   
> company_reference_idlimit 1  ;
>
> ++--+--+
> | company_profile_id | reference_id | reference_source |
> ++--+--+
> |  2 | 331089191| EIN  |
> ++--+--+
>
>  SELECT company_profile_id, url 

Re: Does (into) ever combine 2 numbers into 1?

[lawrence . krubner]

I should remove the parens before they confuse anyone. They are added by 
the log statement. Without the parens:



{:company_profile_id ["2"], :topic :company, :url ["mikeshawauto.com"]}

field-name: 
:company_profile_id

field-value:
["2"]

but I in the final document I only get: 

"final document: " {:company_profile_id ["2"], :topic :company, :url ["
mikeshawauto.com"]}

I want: 

"final document: " {:company_profile_id ["2" "2"], :topic :company, :url ["
mikeshawauto.com"]}



On Wednesday, October 11, 2017 at 1:42:12 PM UTC-4, lawrence...@gmail.com 
wrote:
>
> So I tried this:
>
>   new-field-value (into [] (concat old-field-value 
> field-value))
>
> And I did not get what I expected. Maybe I am sleep deprived, but I don't 
> see why I can't build up a vector with several values. This is in a map in 
> an atom. This is the code: 
>
>   (swap! documents
>  (fn [old-documents]
> (let [
>   document (get old-documents denormalized-id {})
>
>   _ (errors/log document)
>   _ (errors/log field-name)
>   _ (errors/log field-value)
>
>   old-field-value (get old-documents field-name [])
>   new-field-value (into [] (concat old-field-value 
> field-value))
>   document (assoc document field-name new-field-value)
>
>   _ (errors/log "final document: " document)
>   new-documents (assoc old-documents denormalized-id 
> document)
>   ]  
>   new-documents)
>
> Those log statements show me this for old-field-value:
>
> ({:company_profile_id ["2"], :topic :company, :url ["mikeshawauto.com"]})
>
> field-name: 
> (:company_profile_id)
>
> field-value:
> (["2"])
>
> but I in the final document I only get: 
>
> ("final document: " {:company_profile_id ["2"], :topic :company, :url ["
> mikeshawauto.com"]})
>
> I want: 
>
> ("final document: " {:company_profile_id ["2", "2"], :topic :company, :url 
> ["mikeshawauto.com"]})
>
> What am I missing here? 
>
>
>
>
> On Wednesday, October 11, 2017 at 12:28:59 PM UTC-4, lawrence...@gmail.com 
> wrote:
>>
>> So I have this: 
>>
>> ({:company_profile_id ["2"], :topic :company, :url ["mikeshawauto.com"]})
>>
>> And then I get this field name and value: 
>>
>> (:company_profile_id)
>>
>> (["2"])
>>
>> The next 3 lines of code are: 
>>
>>   old-field-value (get old-documents field-name [])
>>   new-field-value (into old-field-value field-value)
>>   document (assoc document field-name new-field-value)
>>
>> And when the function is next called, I end up with :
>>
>> ({:company_profile_id ["2"], :topic :company, :url ["mikeshawauto.com"]})
>>
>> I need: 
>>
>> ({:company_profile_id ["2", "2"], :topic :company, :url ["
>> mikeshawauto.com"]})
>>
>> So perhaps I've misunderstood what (into) is for? How do I add another 
>> the values of these 2 vectors together? 
>>
>>
>>
>>
>>
>> On Wednesday, October 11, 2017 at 12:20:55 PM UTC-4, 
>> lawrence...@gmail.com wrote:
>>>
>>> Using conj instead of into, for no particular reason, except debugging. 
>>> The document is slowly built-up: 
>>>
>>> ({:company_profile_id [["2"]], :topic :company, :url [["mikeshawauto.com
>>> "]]})
>>>
>>> ({:company_profile_id [["2"]], :topic :company, :url [["mikeshawauto.com
>>> "]]})
>>>
>>> ({:company_profile_id [["2"]], :topic :company, :url 
>>> [["mikeshawauto.com"]], 
>>> :reference_id [["331089191"]]})
>>>
>>> In the transition from row 1 to row 2, I assume the field being added is 
>>> company_profile_id. Why is there no change? Why isn't a new value added to 
>>> the vector? I must be missing something obvious. 
>>>
>>>
>>>
>>>
>>>
>>>
>>>
>>> On Wednesday, October 11, 2017 at 11:44:40 AM UTC-4, 
>>> lawrence...@gmail.com wrote:

 I seem unable to figure out where I made a mistake, though this should 
 be simple.

 I have two SQL calls that bring back 5 fields:

  SELECT company_profile_id ,  reference_id,  reference_source   FROM   
 company_reference_idlimit 1  ;

 ++--+--+
 | company_profile_id | reference_id | reference_source |
 ++--+--+
 |  2 | 331089191| EIN  |
 ++--+--+

  SELECT company_profile_id, urlFROM company_website 
 limit 1   ;

 ++--+
 | company_profile_id | url  |
 ++--+
 |  2 | mikeshawauto.com |
 ++--+

 This brings back a total of 5 values. I need to have a document that 
 has 5 values, though if values have the same field-name, then I want to 
 consolidate them into one vector. Ther

Re: Does (into) ever combine 2 numbers into 1?

[lawrence . krubner]

So I tried this:

  new-field-value (into [] (concat old-field-value 
field-value))

And I did not get what I expected. Maybe I am sleep deprived, but I don't 
see why I can't build up a vector with several values. This is in a map in 
an atom. This is the code: 

  (swap! documents
 (fn [old-documents]
(let [
  document (get old-documents denormalized-id {})

  _ (errors/log document)
  _ (errors/log field-name)
  _ (errors/log field-value)

  old-field-value (get old-documents field-name [])
  new-field-value (into [] (concat old-field-value 
field-value))
  document (assoc document field-name new-field-value)

  _ (errors/log "final document: " document)
  new-documents (assoc old-documents denormalized-id 
document)
  ]  
  new-documents)

Those log statements show me this for old-field-value:

({:company_profile_id ["2"], :topic :company, :url ["mikeshawauto.com"]})

field-name: 
(:company_profile_id)

field-value:
(["2"])

but I in the final document I only get: 

("final document: " {:company_profile_id ["2"], :topic :company, :url 
["mikeshawauto.com"]})

I want: 

("final document: " {:company_profile_id ["2", "2"], :topic :company, :url 
["mikeshawauto.com"]})

What am I missing here? 




On Wednesday, October 11, 2017 at 12:28:59 PM UTC-4, lawrence...@gmail.com 
wrote:
>
> So I have this: 
>
> ({:company_profile_id ["2"], :topic :company, :url ["mikeshawauto.com"]})
>
> And then I get this field name and value: 
>
> (:company_profile_id)
>
> (["2"])
>
> The next 3 lines of code are: 
>
>   old-field-value (get old-documents field-name [])
>   new-field-value (into old-field-value field-value)
>   document (assoc document field-name new-field-value)
>
> And when the function is next called, I end up with :
>
> ({:company_profile_id ["2"], :topic :company, :url ["mikeshawauto.com"]})
>
> I need: 
>
> ({:company_profile_id ["2", "2"], :topic :company, :url ["mikeshawauto.com
> "]})
>
> So perhaps I've misunderstood what (into) is for? How do I add another the 
> values of these 2 vectors together? 
>
>
>
>
>
> On Wednesday, October 11, 2017 at 12:20:55 PM UTC-4, lawrence...@gmail.com 
> wrote:
>>
>> Using conj instead of into, for no particular reason, except debugging. 
>> The document is slowly built-up: 
>>
>> ({:company_profile_id [["2"]], :topic :company, :url [["mikeshawauto.com
>> "]]})
>>
>> ({:company_profile_id [["2"]], :topic :company, :url [["mikeshawauto.com
>> "]]})
>>
>> ({:company_profile_id [["2"]], :topic :company, :url [["mikeshawauto.com"]], 
>> :reference_id [["331089191"]]})
>>
>> In the transition from row 1 to row 2, I assume the field being added is 
>> company_profile_id. Why is there no change? Why isn't a new value added to 
>> the vector? I must be missing something obvious. 
>>
>>
>>
>>
>>
>>
>>
>> On Wednesday, October 11, 2017 at 11:44:40 AM UTC-4, 
>> lawrence...@gmail.com wrote:
>>>
>>> I seem unable to figure out where I made a mistake, though this should 
>>> be simple.
>>>
>>> I have two SQL calls that bring back 5 fields:
>>>
>>>  SELECT company_profile_id ,  reference_id,  reference_source   FROM   
>>> company_reference_idlimit 1  ;
>>>
>>> ++--+--+
>>> | company_profile_id | reference_id | reference_source |
>>> ++--+--+
>>> |  2 | 331089191| EIN  |
>>> ++--+--+
>>>
>>>  SELECT company_profile_id, urlFROM company_website 
>>> limit 1   ;
>>>
>>> ++--+
>>> | company_profile_id | url  |
>>> ++--+
>>> |  2 | mikeshawauto.com |
>>> ++--+
>>>
>>> This brings back a total of 5 values. I need to have a document that has 
>>> 5 values, though if values have the same field-name, then I want to 
>>> consolidate them into one vector. There are 4 unique field names, so I 
>>> expect to end up with 4 vectors, holding 5 values. Instead, I get this: 
>>>
>>> ({:company_profile_id ["2"], :topic :company, 
>>> :how-many-rows-and-fields-from-database 13, :url ["mikeshawauto.com"], 
>>> :reference_id ["331089191"], :reference_source ["ein"]})
>>>
>>> I expect: 
>>>
>>> {:company_profile_id ["2" "2"]
>>>
>>> but I get: 
>>>
>>> {:company_profile_id ["2"]
>>>
>>> The documents are combined in a map in an atom, with this function: 
>>>
>>>
>>> (def documents (atom {}))
>>>
>>>
>>> (defn update-documents
>>>   [denormalized-id field-name field-value 
>>> how-many-rows-and-fields-from-database topic]
>>>   {:pre [
>>>  (not (nil? denormalized-id))
>>>  (not (nil? field-name))

Re: Does (into) ever combine 2 numbers into 1?

[lawrence . krubner]

So I have this: 

({:company_profile_id ["2"], :topic :company, :url ["mikeshawauto.com"]})

And then I get this field name and value: 

(:company_profile_id)

(["2"])

The next 3 lines of code are: 

  old-field-value (get old-documents field-name [])
  new-field-value (into old-field-value field-value)
  document (assoc document field-name new-field-value)

And when the function is next called, I end up with :

({:company_profile_id ["2"], :topic :company, :url ["mikeshawauto.com"]})

I need: 

({:company_profile_id ["2", "2"], :topic :company, :url 
["mikeshawauto.com"]})

So perhaps I've misunderstood what (into) is for? How do I add another the 
values of these 2 vectors together? 





On Wednesday, October 11, 2017 at 12:20:55 PM UTC-4, lawrence...@gmail.com 
wrote:
>
> Using conj instead of into, for no particular reason, except debugging. 
> The document is slowly built-up: 
>
> ({:company_profile_id [["2"]], :topic :company, :url [["mikeshawauto.com
> "]]})
>
> ({:company_profile_id [["2"]], :topic :company, :url [["mikeshawauto.com
> "]]})
>
> ({:company_profile_id [["2"]], :topic :company, :url [["mikeshawauto.com"]], 
> :reference_id [["331089191"]]})
>
> In the transition from row 1 to row 2, I assume the field being added is 
> company_profile_id. Why is there no change? Why isn't a new value added to 
> the vector? I must be missing something obvious. 
>
>
>
>
>
>
>
> On Wednesday, October 11, 2017 at 11:44:40 AM UTC-4, lawrence...@gmail.com 
> wrote:
>>
>> I seem unable to figure out where I made a mistake, though this should be 
>> simple.
>>
>> I have two SQL calls that bring back 5 fields:
>>
>>  SELECT company_profile_id ,  reference_id,  reference_source   FROM   
>> company_reference_idlimit 1  ;
>>
>> ++--+--+
>> | company_profile_id | reference_id | reference_source |
>> ++--+--+
>> |  2 | 331089191| EIN  |
>> ++--+--+
>>
>>  SELECT company_profile_id, urlFROM company_website limit 
>> 1   ;
>>
>> ++--+
>> | company_profile_id | url  |
>> ++--+
>> |  2 | mikeshawauto.com |
>> ++--+
>>
>> This brings back a total of 5 values. I need to have a document that has 
>> 5 values, though if values have the same field-name, then I want to 
>> consolidate them into one vector. There are 4 unique field names, so I 
>> expect to end up with 4 vectors, holding 5 values. Instead, I get this: 
>>
>> ({:company_profile_id ["2"], :topic :company, 
>> :how-many-rows-and-fields-from-database 13, :url ["mikeshawauto.com"], 
>> :reference_id ["331089191"], :reference_source ["ein"]})
>>
>> I expect: 
>>
>> {:company_profile_id ["2" "2"]
>>
>> but I get: 
>>
>> {:company_profile_id ["2"]
>>
>> The documents are combined in a map in an atom, with this function: 
>>
>>
>> (def documents (atom {}))
>>
>>
>> (defn update-documents
>>   [denormalized-id field-name field-value 
>> how-many-rows-and-fields-from-database topic]
>>   {:pre [
>>  (not (nil? denormalized-id))
>>  (not (nil? field-name))
>>  (vector? field-value)
>>  ]}
>>   (swap! documents
>>  (fn [old-documents]
>>(slingshot/try+
>> (let [
>>   document (get old-documents denormalized-id {})
>>   old-field-value (get old-documents field-name [])
>>   new-field-value (into old-field-value field-value)
>>   document (assoc document field-name new-field-value)
>>
>>   previous-how-many-rows-and-fields-from-database (get 
>> document :how-many-rows-and-fields-from-database 0)
>>   new-how-many-rows-and-fields-from-database (+ 
>> previous-how-many-rows-and-fields-from-database 
>> how-many-rows-and-fields-from-database)
>>   document (assoc document :topic topic)
>>   document (assoc document 
>> :how-many-rows-and-fields-from-database 
>> new-how-many-rows-and-fields-from-database)
>>   new-documents (assoc old-documents denormalized-id 
>> document)
>>   ]
>>   new-documents)
>> (catch Object o
>>   (errors/log o) 
>>   (slingshot/throw+ {
>>  :type ::update-documents
>>  :error o
>>  }
>> ))
>>
>> Can I assume that this always gives me 2 values in a vector? 
>>
>>   old-field-value (get old-documents field-name [])
>>   new-field-value (into old-field-value field-value)
>>   document (assoc document field-name new-field-value)
>>

Re: Does (into) ever combine 2 numbers into 1?

[lawrence . krubner]

Using conj instead of into, for no particular reason, except debugging. The 
document is slowly built-up: 

({:company_profile_id [["2"]], :topic :company, :url 
[["mikeshawauto.com"]]})

({:company_profile_id [["2"]], :topic :company, :url 
[["mikeshawauto.com"]]})

({:company_profile_id [["2"]], :topic :company, :url 
[["mikeshawauto.com"]], :reference_id [["331089191"]]})

In the transition from row 1 to row 2, I assume the field being added is 
company_profile_id. Why is there no change? Why isn't a new value added to 
the vector? I must be missing something obvious. 







On Wednesday, October 11, 2017 at 11:44:40 AM UTC-4, lawrence...@gmail.com 
wrote:
>
> I seem unable to figure out where I made a mistake, though this should be 
> simple.
>
> I have two SQL calls that bring back 5 fields:
>
>  SELECT company_profile_id ,  reference_id,  reference_source   FROM   
> company_reference_idlimit 1  ;
>
> ++--+--+
> | company_profile_id | reference_id | reference_source |
> ++--+--+
> |  2 | 331089191| EIN  |
> ++--+--+
>
>  SELECT company_profile_id, urlFROM company_website limit 
> 1   ;
>
> ++--+
> | company_profile_id | url  |
> ++--+
> |  2 | mikeshawauto.com |
> ++--+
>
> This brings back a total of 5 values. I need to have a document that has 5 
> values, though if values have the same field-name, then I want to 
> consolidate them into one vector. There are 4 unique field names, so I 
> expect to end up with 4 vectors, holding 5 values. Instead, I get this: 
>
> ({:company_profile_id ["2"], :topic :company, 
> :how-many-rows-and-fields-from-database 13, :url ["mikeshawauto.com"], 
> :reference_id ["331089191"], :reference_source ["ein"]})
>
> I expect: 
>
> {:company_profile_id ["2" "2"]
>
> but I get: 
>
> {:company_profile_id ["2"]
>
> The documents are combined in a map in an atom, with this function: 
>
>
> (def documents (atom {}))
>
>
> (defn update-documents
>   [denormalized-id field-name field-value 
> how-many-rows-and-fields-from-database topic]
>   {:pre [
>  (not (nil? denormalized-id))
>  (not (nil? field-name))
>  (vector? field-value)
>  ]}
>   (swap! documents
>  (fn [old-documents]
>(slingshot/try+
> (let [
>   document (get old-documents denormalized-id {})
>   old-field-value (get old-documents field-name [])
>   new-field-value (into old-field-value field-value)
>   document (assoc document field-name new-field-value)
>
>   previous-how-many-rows-and-fields-from-database (get 
> document :how-many-rows-and-fields-from-database 0)
>   new-how-many-rows-and-fields-from-database (+ 
> previous-how-many-rows-and-fields-from-database 
> how-many-rows-and-fields-from-database)
>   document (assoc document :topic topic)
>   document (assoc document 
> :how-many-rows-and-fields-from-database 
> new-how-many-rows-and-fields-from-database)
>   new-documents (assoc old-documents denormalized-id 
> document)
>   ]
>   new-documents)
> (catch Object o
>   (errors/log o) 
>   (slingshot/throw+ {
>  :type ::update-documents
>  :error o
>  }
> ))
>
> Can I assume that this always gives me 2 values in a vector? 
>
>   old-field-value (get old-documents field-name [])
>   new-field-value (into old-field-value field-value)
>   document (assoc document field-name new-field-value)
>
> I'm going to guess that the bug is somewhere else in my code. But if 
> anyone sees a flaw in this function, I'd be grateful if you could point it 
> out to me. 
>
>
>
>

-- 
You received this message because you are subscribed to the Google
Groups "Clojure" group.
To post to this group, send email to clojure@googlegroups.com
Note that posts from new members are moderated - please be patient with your 
first post.
To unsubscribe from this group, send email to
clojure+unsubscr...@googlegroups.com
For more options, visit this group at
http://groups.google.com/group/clojure?hl=en
--- 
You received this message because you are subscribed to the Google Groups 
"Clojure" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to clojure+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: Does (into) ever combine 2 numbers into 1?

[lawrence . krubner]

Nevermind. Something about my reasoning is off on this one. I notice that 
even if I use conj, the same problem happens, I get 4 values in 4 vectors 
instead of 5 values in 4 vectors. So there must be some earlier step that 
I've gotten wrong. 


On Wednesday, October 11, 2017 at 11:44:40 AM UTC-4, lawrence...@gmail.com 
wrote:
>
> I seem unable to figure out where I made a mistake, though this should be 
> simple.
>
> I have two SQL calls that bring back 5 fields:
>
>  SELECT company_profile_id ,  reference_id,  reference_source   FROM   
> company_reference_idlimit 1  ;
>
> ++--+--+
> | company_profile_id | reference_id | reference_source |
> ++--+--+
> |  2 | 331089191| EIN  |
> ++--+--+
>
>  SELECT company_profile_id, urlFROM company_website limit 
> 1   ;
>
> ++--+
> | company_profile_id | url  |
> ++--+
> |  2 | mikeshawauto.com |
> ++--+
>
> This brings back a total of 5 values. I need to have a document that has 5 
> values, though if values have the same field-name, then I want to 
> consolidate them into one vector. There are 4 unique field names, so I 
> expect to end up with 4 vectors, holding 5 values. Instead, I get this: 
>
> ({:company_profile_id ["2"], :topic :company, 
> :how-many-rows-and-fields-from-database 13, :url ["mikeshawauto.com"], 
> :reference_id ["331089191"], :reference_source ["ein"]})
>
> I expect: 
>
> {:company_profile_id ["2" "2"]
>
> but I get: 
>
> {:company_profile_id ["2"]
>
> The documents are combined in a map in an atom, with this function: 
>
>
> (def documents (atom {}))
>
>
> (defn update-documents
>   [denormalized-id field-name field-value 
> how-many-rows-and-fields-from-database topic]
>   {:pre [
>  (not (nil? denormalized-id))
>  (not (nil? field-name))
>  (vector? field-value)
>  ]}
>   (swap! documents
>  (fn [old-documents]
>(slingshot/try+
> (let [
>   document (get old-documents denormalized-id {})
>   old-field-value (get old-documents field-name [])
>   new-field-value (into old-field-value field-value)
>   document (assoc document field-name new-field-value)
>
>   previous-how-many-rows-and-fields-from-database (get 
> document :how-many-rows-and-fields-from-database 0)
>   new-how-many-rows-and-fields-from-database (+ 
> previous-how-many-rows-and-fields-from-database 
> how-many-rows-and-fields-from-database)
>   document (assoc document :topic topic)
>   document (assoc document 
> :how-many-rows-and-fields-from-database 
> new-how-many-rows-and-fields-from-database)
>   new-documents (assoc old-documents denormalized-id 
> document)
>   ]
>   new-documents)
> (catch Object o
>   (errors/log o) 
>   (slingshot/throw+ {
>  :type ::update-documents
>  :error o
>  }
> ))
>
> Can I assume that this always gives me 2 values in a vector? 
>
>   old-field-value (get old-documents field-name [])
>   new-field-value (into old-field-value field-value)
>   document (assoc document field-name new-field-value)
>
> I'm going to guess that the bug is somewhere else in my code. But if 
> anyone sees a flaw in this function, I'd be grateful if you could point it 
> out to me. 
>
>
>
>

-- 
You received this message because you are subscribed to the Google
Groups "Clojure" group.
To post to this group, send email to clojure@googlegroups.com
Note that posts from new members are moderated - please be patient with your 
first post.
To unsubscribe from this group, send email to
clojure+unsubscr...@googlegroups.com
For more options, visit this group at
http://groups.google.com/group/clojure?hl=en
--- 
You received this message because you are subscribed to the Google Groups 
"Clojure" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to clojure+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Does (into) ever combine 2 numbers into 1?

[lawrence . krubner]

I seem unable to figure out where I made a mistake, though this should be 
simple.

I have two SQL calls that bring back 5 fields:

 SELECT company_profile_id ,  reference_id,  reference_source   FROM   
company_reference_idlimit 1  ;

++--+--+
| company_profile_id | reference_id | reference_source |
++--+--+
|  2 | 331089191| EIN  |
++--+--+

 SELECT company_profile_id, urlFROM company_website limit 1 
  ;

++--+
| company_profile_id | url  |
++--+
|  2 | mikeshawauto.com |
++--+

This brings back a total of 5 values. I need to have a document that has 5 
values, though if values have the same field-name, then I want to 
consolidate them into one vector. There are 4 unique field names, so I 
expect to end up with 4 vectors, holding 5 values. Instead, I get this: 

({:company_profile_id ["2"], :topic :company, 
:how-many-rows-and-fields-from-database 13, :url ["mikeshawauto.com"], 
:reference_id ["331089191"], :reference_source ["ein"]})

I expect: 

{:company_profile_id ["2" "2"]

but I get: 

{:company_profile_id ["2"]

The documents are combined in a map in an atom, with this function: 


(def documents (atom {}))


(defn update-documents
  [denormalized-id field-name field-value 
how-many-rows-and-fields-from-database topic]
  {:pre [
 (not (nil? denormalized-id))
 (not (nil? field-name))
 (vector? field-value)
 ]}
  (swap! documents
 (fn [old-documents]
   (slingshot/try+
(let [
  document (get old-documents denormalized-id {})
  old-field-value (get old-documents field-name [])
  new-field-value (into old-field-value field-value)
  document (assoc document field-name new-field-value)

  previous-how-many-rows-and-fields-from-database (get 
document :how-many-rows-and-fields-from-database 0)
  new-how-many-rows-and-fields-from-database (+ 
previous-how-many-rows-and-fields-from-database 
how-many-rows-and-fields-from-database)
  document (assoc document :topic topic)
  document (assoc document 
:how-many-rows-and-fields-from-database 
new-how-many-rows-and-fields-from-database)
  new-documents (assoc old-documents denormalized-id 
document)
  ]
  new-documents)
(catch Object o
  (errors/log o) 
  (slingshot/throw+ {
 :type ::update-documents
 :error o
 }
))

Can I assume that this always gives me 2 values in a vector? 

  old-field-value (get old-documents field-name [])
  new-field-value (into old-field-value field-value)
  document (assoc document field-name new-field-value)

I'm going to guess that the bug is somewhere else in my code. But if anyone 
sees a flaw in this function, I'd be grateful if you could point it out to 
me. 



-- 
You received this message because you are subscribed to the Google
Groups "Clojure" group.
To post to this group, send email to clojure@googlegroups.com
Note that posts from new members are moderated - please be patient with your 
first post.
To unsubscribe from this group, send email to
clojure+unsubscr...@googlegroups.com
For more options, visit this group at
http://groups.google.com/group/clojure?hl=en
--- 
You received this message because you are subscribed to the Google Groups 
"Clojure" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to clojure+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: Can slingshot/try+ and then catch Object really catch any error?

[lawrence . krubner]

I figured a common reason for error messages to disappear is when there is 
another Exception inside of the catch block, so the catch block itself dies 
without doing anything. 

I had copy-and-pasted this from somewhere, without looking at it much:

   (catch Exception e
 (println e)
 (stack/parse-exception e)
 (throw (ex-info "The app has died!")))

It took me awhile before I saw this: 

:type clojure.lang.ArityException
   :message Wrong number of args (1) passed to: core/ex-info
   :at [clojure.lang.AFn throwArity AFn.java 429]}]

Looking here reveals that there is no one argument version of ex-info:

https://clojuredocs.org/clojure.core/ex-info

My fault, obviously, for being careless when mixing different kinds of 
error reporting. 


On Monday, October 9, 2017 at 2:59:35 PM UTC-4, Sean Corfield wrote:
>
> Timbre has a logged-future macro that you can use instead of future so 
> that exceptions are automatically logged.
>
>  
>
> Sean Corfield -- (970) FOR-SEAN -- (904) 302-SEAN
> An Architect's View -- http://corfield.org/
>
> "If you're not annoying somebody, you're not really alive."
> -- Margaret Atwood
>
>  
> --
> *From:* clo...@googlegroups.com   > on behalf of lawrence...@gmail.com  <
> lawrence...@gmail.com >
> *Sent:* Monday, October 9, 2017 11:23:05 AM
> *To:* Clojure
> *Subject:* Re: Can slingshot/try+ and then catch Object really catch any 
> error? 
>  
> MatchingSocks, thanks for that. I think the pattern I followed everywhere 
> was: 
>
> (future
>   (slingshot/try+
>  ;;; some code
> (catch Object o
>(println o
>
> So I think I do catch everything inside of each future that I launch. But 
> I will check again. Perhaps I missed one somewhere.
>
>
> On Monday, October 9, 2017 at 8:12:33 AM UTC-4, Matching Socks wrote: 
>>
>> The linked page 
>> https://stuartsierra.com/2015/05/27/clojure-uncaught-exceptions also 
>> says "Another wrinkle: exceptions inside a future are always caught by the 
>> Future. The exception will not be thrown until something calls Future.get 
>> (deref in Clojure)."  So you would need to review the pattern of "(try 
>> (future (catch (throw".
>>
> -- 
> You received this message because you are subscribed to the Google
> Groups "Clojure" group.
> To post to this group, send email to clo...@googlegroups.com 
> Note that posts from new members are moderated - please be patient with 
> your first post.
> To unsubscribe from this group, send email to
> clojure+u...@googlegroups.com 
> For more options, visit this group at
> http://groups.google.com/group/clojure?hl=en
> --- 
> You received this message because you are subscribed to the Google Groups 
> "Clojure" group.
> To unsubscribe from this group and stop receiving emails from it, send an 
> email to clojure+u...@googlegroups.com .
> For more options, visit https://groups.google.com/d/optout.
>

-- 
You received this message because you are subscribed to the Google
Groups "Clojure" group.
To post to this group, send email to clojure@googlegroups.com
Note that posts from new members are moderated - please be patient with your 
first post.
To unsubscribe from this group, send email to
clojure+unsubscr...@googlegroups.com
For more options, visit this group at
http://groups.google.com/group/clojure?hl=en
--- 
You received this message because you are subscribed to the Google Groups 
"Clojure" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to clojure+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: Can slingshot/try+ and then catch Object really catch any error?

[lawrence . krubner]

Sean Corfield, thank you, that is a great tip.


On Monday, October 9, 2017 at 2:59:35 PM UTC-4, Sean Corfield wrote:
>
> Timbre has a logged-future macro that you can use instead of future so 
> that exceptions are automatically logged.
>
>  
>
> Sean Corfield -- (970) FOR-SEAN -- (904) 302-SEAN
> An Architect's View -- http://corfield.org/
>
> "If you're not annoying somebody, you're not really alive."
> -- Margaret Atwood
>
>  
> --
> *From:* clo...@googlegroups.com   > on behalf of lawrence...@gmail.com  <
> lawrence...@gmail.com >
> *Sent:* Monday, October 9, 2017 11:23:05 AM
> *To:* Clojure
> *Subject:* Re: Can slingshot/try+ and then catch Object really catch any 
> error? 
>  
> MatchingSocks, thanks for that. I think the pattern I followed everywhere 
> was: 
>
> (future
>   (slingshot/try+
>  ;;; some code
> (catch Object o
>(println o
>
> So I think I do catch everything inside of each future that I launch. But 
> I will check again. Perhaps I missed one somewhere.
>
>
> On Monday, October 9, 2017 at 8:12:33 AM UTC-4, Matching Socks wrote: 
>>
>> The linked page 
>> https://stuartsierra.com/2015/05/27/clojure-uncaught-exceptions also 
>> says "Another wrinkle: exceptions inside a future are always caught by the 
>> Future. The exception will not be thrown until something calls Future.get 
>> (deref in Clojure)."  So you would need to review the pattern of "(try 
>> (future (catch (throw".
>>
> -- 
> You received this message because you are subscribed to the Google
> Groups "Clojure" group.
> To post to this group, send email to clo...@googlegroups.com 
> Note that posts from new members are moderated - please be patient with 
> your first post.
> To unsubscribe from this group, send email to
> clojure+u...@googlegroups.com 
> For more options, visit this group at
> http://groups.google.com/group/clojure?hl=en
> --- 
> You received this message because you are subscribed to the Google Groups 
> "Clojure" group.
> To unsubscribe from this group and stop receiving emails from it, send an 
> email to clojure+u...@googlegroups.com .
> For more options, visit https://groups.google.com/d/optout.
>

-- 
You received this message because you are subscribed to the Google
Groups "Clojure" group.
To post to this group, send email to clojure@googlegroups.com
Note that posts from new members are moderated - please be patient with your 
first post.
To unsubscribe from this group, send email to
clojure+unsubscr...@googlegroups.com
For more options, visit this group at
http://groups.google.com/group/clojure?hl=en
--- 
You received this message because you are subscribed to the Google Groups 
"Clojure" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to clojure+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: Can slingshot/try+ and then catch Object really catch any error?

[lawrence . krubner]

Gary Verhaegen, thanks for that. The idea of buffers and threads dying is a 
good one and gives me something to look for. 


On Monday, October 9, 2017 at 2:23:05 PM UTC-4, lawrence...@gmail.com wrote:
>
> MatchingSocks, thanks for that. I think the pattern I followed everywhere 
> was:
>
> (future
>   (slingshot/try+
>  ;;; some code
> (catch Object o
>(println o
>
> So I think I do catch everything inside of each future that I launch. But 
> I will check again. Perhaps I missed one somewhere.
>
>
> On Monday, October 9, 2017 at 8:12:33 AM UTC-4, Matching Socks wrote:
>>
>> The linked page 
>> https://stuartsierra.com/2015/05/27/clojure-uncaught-exceptions also 
>> says "Another wrinkle: exceptions inside a future are always caught by the 
>> Future. The exception will not be thrown until something calls Future.get 
>> (deref in Clojure)."  So you would need to review the pattern of "(try 
>> (future (catch (throw".
>>
>

-- 
You received this message because you are subscribed to the Google
Groups "Clojure" group.
To post to this group, send email to clojure@googlegroups.com
Note that posts from new members are moderated - please be patient with your 
first post.
To unsubscribe from this group, send email to
clojure+unsubscr...@googlegroups.com
For more options, visit this group at
http://groups.google.com/group/clojure?hl=en
--- 
You received this message because you are subscribed to the Google Groups 
"Clojure" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to clojure+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: Can slingshot/try+ and then catch Object really catch any error?

[lawrence . krubner]

MatchingSocks, thanks for that. I think the pattern I followed everywhere 
was:

(future
  (slingshot/try+
 ;;; some code
(catch Object o
   (println o

So I think I do catch everything inside of each future that I launch. But I 
will check again. Perhaps I missed one somewhere.


On Monday, October 9, 2017 at 8:12:33 AM UTC-4, Matching Socks wrote:
>
> The linked page 
> https://stuartsierra.com/2015/05/27/clojure-uncaught-exceptions also says 
> "Another wrinkle: exceptions inside a future are always caught by the 
> Future. The exception will not be thrown until something calls Future.get 
> (deref in Clojure)."  So you would need to review the pattern of "(try 
> (future (catch (throw".
>

-- 
You received this message because you are subscribed to the Google
Groups "Clojure" group.
To post to this group, send email to clojure@googlegroups.com
Note that posts from new members are moderated - please be patient with your 
first post.
To unsubscribe from this group, send email to
clojure+unsubscr...@googlegroups.com
For more options, visit this group at
http://groups.google.com/group/clojure?hl=en
--- 
You received this message because you are subscribed to the Google Groups 
"Clojure" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to clojure+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: Can slingshot/try+ and then catch Object really catch any error?

[lawrence . krubner]

Shantanu Kumar, thanks for that, I might try it. I assume you've never had 
the problem I'm talking about, of messages on background threads that 
disappear?


On Monday, October 9, 2017 at 2:56:24 AM UTC-4, Shantanu Kumar wrote:
>
>
>>
>> I'm curious what others do for logging? 
>>
>
> At Concur we using Cambium https://github.com/kumarshantanu/cambium 
> that's being moved (WIP) here: https://github.com/cambium-clojure
>
> Cambium wraps SLF4j and gives a Clojure API (which extends tools.logging) 
> to use it's MDC feature.
>
>
> Shantanu
>
>>

-- 
You received this message because you are subscribed to the Google
Groups "Clojure" group.
To post to this group, send email to clojure@googlegroups.com
Note that posts from new members are moderated - please be patient with your 
first post.
To unsubscribe from this group, send email to
clojure+unsubscr...@googlegroups.com
For more options, visit this group at
http://groups.google.com/group/clojure?hl=en
--- 
You received this message because you are subscribed to the Google Groups 
"Clojure" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to clojure+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: Can slingshot/try+ and then catch Object really catch any error?

[lawrence . krubner]

Kumar,

Just so you know, on this page:

https://github.com/cambium-clojure/cambium.logback.core

you link to here:

https://cambium-clojure.github.io/

but I get a 404 when I go there. 



On Monday, October 9, 2017 at 2:56:24 AM UTC-4, Shantanu Kumar wrote:
>
>
>>
>> I'm curious what others do for logging? 
>>
>
> At Concur we using Cambium https://github.com/kumarshantanu/cambium 
> that's being moved (WIP) here: https://github.com/cambium-clojure
>
> Cambium wraps SLF4j and gives a Clojure API (which extends tools.logging) 
> to use it's MDC feature.
>
>
> Shantanu
>
>>

-- 
You received this message because you are subscribed to the Google
Groups "Clojure" group.
To post to this group, send email to clojure@googlegroups.com
Note that posts from new members are moderated - please be patient with your 
first post.
To unsubscribe from this group, send email to
clojure+unsubscr...@googlegroups.com
For more options, visit this group at
http://groups.google.com/group/clojure?hl=en
--- 
You received this message because you are subscribed to the Google Groups 
"Clojure" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to clojure+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: Can slingshot/try+ and then catch Object really catch any error?

[lawrence . krubner]

I'm using Cider in Emacs, but the problems I face only occur when I've 
created an uberjar and I try to run the app on the server. In the REPL I 
only work with small amounts of data, whereas on the server I work with a 
few million. Not sure how to get around that as I'm working from my 
apartment and my client has AWS roles set up so that the database can only 
be reached from the server.

I'm fine with writing everything to a log file, if that would catch 
messages from background threads, though I'm curious what setups others 
use? I'm a bit surprised I can't get Timbre to work the way I want to, 
though I have not put much effort into configuring it to write to a log 
file. In the past I've been happy to have it write to stdout and then (in 
production) Supervisord would take the stdout and write it to a log file. 

I'm curious what others do for logging? 


On Monday, October 9, 2017 at 12:31:16 AM UTC-4, tbc++ wrote:
>
> What REPL are you using? A rather nasty side-effect of NRepl is that it 
> tends to send messages from non-main-threads to weird places. 
>
> On Sun, Oct 8, 2017 at 10:29 PM, > 
> wrote:
>
>> I just re-wrote much of the code to run on the main thread. And so now I 
>> can see the error message (which was about a nil value in a place where I 
>> was certain there would be no nil values). Which is great. But since the 
>> app does a lot of I/O, I would like to for this app to be multi-threaded. 
>> But I need to figure out a way that I can see errors that happen in 
>> background threads. 
>>
>>
>> On Sunday, October 8, 2017 at 11:46:26 PM UTC-4, tbc++ wrote:
>>>
>>> I don't think you can catch an Object on the JVM can you? What happens 
>>> if you catch Throwable?
>>>
>>> On Sun, Oct 8, 2017 at 9:43 PM,  wrote:
>>>
 So, for instance, this says that 10 documents were retried:

 {"message" {:num-slabs 1, :num-active-slabs 1, :enqueued 389, :retried 
 10, :completed 378, :in-progress 1}}

 The only place that I call retry! is in this catch block (this function 
 runs in a background thread):

 (defn advance
   [message]
   {:pre [
  (= (type message) durable_queue.Task)
  ]}
   (slingshot/try+
(doseq [[k v] @message]
  (when-not (or
 (= :how-many-rows-and-fields-from-database k)
 (= :database-table k)
 (= :denormalized-id k)
 (= :topic k))
(transform
 (get @message :denormalized-id :missing-denormalized-id)
 k
 v
 (get @message :topic :missing-topic)
 (get @message :how-many-rows-and-fields-from-database 
 :missing-how-many-rows-and-fields-from-database

(durable/complete! message)
(catch Object o
  (durable/retry! message)
  (slingshot/throw+ {
 :type ::advance
 :error o
 }


 So clearly, the code has a problem with some of the documents, and so 
 an error is thrown, and it is caught by the catch block. I would love to 
 see the error. I can not. If I put println in the catch block, the message 
 never shows up in the terminal. Likewise if I use Timbre. 

 I would love to see the error object. My inability to see the error 
 objects has crippled my productivity. 

 Can anyone tell me what I need to do to see these errors? 





 On Sunday, October 8, 2017 at 11:13:10 PM UTC-4, lawrence...@gmail.com 
 wrote:

> I'm trying to use a Zach Tellman's durable-queue to pass information 
> from one part of my app to another. I have 3 queues. I also have a 
> function 
> that runs every 30 seconds, that is suppose to report some statistics for 
> me: 
>
>   (fn []
> (future
>   (slingshot/try+
>(errors/log "Resource usage: "
>   
>  (monitoring/show-stats-regarding-resources-used-by-this-app))
>(errors/log "Stats about 
> from-mysql-to-tables-queue: " (durable/stats from-mysql-to-tables-queue))
>(errors/log "Stats about 
> from-tables-to-topics-queue: " (durable/stats 
> from-tables-to-topics-queue))
>(errors/log "Stats about the 
> from-topics-to-persistence-queue: " (durable/stats 
> from-topics-to-persistence-queue))
>(catch Object o
>  (slingshot/throw+ {
> :type 
> ::cycle-resource-usage-report
> :error o
> }
>)
>
> I think there is an error here, because I see

Re: Can slingshot/try+ and then catch Object really catch any error?

[lawrence . krubner]

I just re-wrote much of the code to run on the main thread. And so now I 
can see the error message (which was about a nil value in a place where I 
was certain there would be no nil values). Which is great. But since the 
app does a lot of I/O, I would like to for this app to be multi-threaded. 
But I need to figure out a way that I can see errors that happen in 
background threads. 


On Sunday, October 8, 2017 at 11:46:26 PM UTC-4, tbc++ wrote:
>
> I don't think you can catch an Object on the JVM can you? What happens if 
> you catch Throwable?
>
> On Sun, Oct 8, 2017 at 9:43 PM, > 
> wrote:
>
>> So, for instance, this says that 10 documents were retried:
>>
>> {"message" {:num-slabs 1, :num-active-slabs 1, :enqueued 389, :retried 
>> 10, :completed 378, :in-progress 1}}
>>
>> The only place that I call retry! is in this catch block (this function 
>> runs in a background thread):
>>
>> (defn advance
>>   [message]
>>   {:pre [
>>  (= (type message) durable_queue.Task)
>>  ]}
>>   (slingshot/try+
>>(doseq [[k v] @message]
>>  (when-not (or
>> (= :how-many-rows-and-fields-from-database k)
>> (= :database-table k)
>> (= :denormalized-id k)
>> (= :topic k))
>>(transform
>> (get @message :denormalized-id :missing-denormalized-id)
>> k
>> v
>> (get @message :topic :missing-topic)
>> (get @message :how-many-rows-and-fields-from-database 
>> :missing-how-many-rows-and-fields-from-database
>>
>>(durable/complete! message)
>>(catch Object o
>>  (durable/retry! message)
>>  (slingshot/throw+ {
>> :type ::advance
>> :error o
>> }
>>
>>
>> So clearly, the code has a problem with some of the documents, and so an 
>> error is thrown, and it is caught by the catch block. I would love to see 
>> the error. I can not. If I put println in the catch block, the message 
>> never shows up in the terminal. Likewise if I use Timbre. 
>>
>> I would love to see the error object. My inability to see the error 
>> objects has crippled my productivity. 
>>
>> Can anyone tell me what I need to do to see these errors? 
>>
>>
>>
>>
>>
>> On Sunday, October 8, 2017 at 11:13:10 PM UTC-4, lawrence...@gmail.com 
>> wrote:
>>
>>> I'm trying to use a Zach Tellman's durable-queue to pass information 
>>> from one part of my app to another. I have 3 queues. I also have a function 
>>> that runs every 30 seconds, that is suppose to report some statistics for 
>>> me: 
>>>
>>>   (fn []
>>> (future
>>>   (slingshot/try+
>>>(errors/log "Resource usage: "
>>>   
>>>  (monitoring/show-stats-regarding-resources-used-by-this-app))
>>>(errors/log "Stats about from-mysql-to-tables-queue: 
>>> " (durable/stats from-mysql-to-tables-queue))
>>>(errors/log "Stats about from-tables-to-topics-queue: 
>>> " (durable/stats from-tables-to-topics-queue))
>>>(errors/log "Stats about the 
>>> from-topics-to-persistence-queue: " (durable/stats 
>>> from-topics-to-persistence-queue))
>>>(catch Object o
>>>  (slingshot/throw+ {
>>> :type 
>>> ::cycle-resource-usage-report
>>> :error o
>>> }
>>>)
>>>
>>> I think there is an error here, because I see the first (errors/log) but 
>>> typically not anything after that. Can I rely on the fact that this should 
>>> catch anything: 
>>>
>>>(catch Object o
>>>
>>> Because it doesn't seem to work. Instead, the app runs for 8 minutes, 
>>> then prints data about the queues just once, then never again. Assuming an 
>>> error is occuring, why can't I see the error? 
>>>
>>>
>>> {:timestamp "17-10-09 01:27:45", :level :trace, :hostname 
>>> "ip-172-31-4-54", :message ("STARTING")}
>>>
>>> Loading class `com.mysql.jdbc.Driver'. This is deprecated. The new 
>>> driver class is `com.mysql.cj.jdbc.Driver'. The driver is automatically 
>>> registered via the SPI and manual loading of the driver class is generally 
>>> unnecessary.
>>>
>>>
>>> {:timestamp "17-10-09 01:28:16", :level :trace, :hostname 
>>> "ip-172-31-4-54", :message ("\n\n\n\n\nResource usage: " "Memory in use 
>>> (percentage/used/max-heap): (\"67%\" \"2414M\" \"3568M\")\n\nCPU usage 
>>> (how-many-cpu's/load-average):  [4 1.56]\n\nFree memory in jvm: 
>>> [1201758848]")}
>>>
>>>
>>> {:timestamp "17-10-09 01:28:49", :level :trace, :hostname 
>>> "ip-172-31-4-54", :message ("\n\n\n\n\nResource usage: " "Memory in use 
>>> (percentage/used/max-heap): (\"81%\" \"2900M\" \"3568M\")\n\nCPU usage 
>>> (how-many-cpu's/load-average):  [4 2.64]\n\nFr

Re: Can slingshot/try+ and then catch Object really catch any error?

[lawrence . krubner]

Possibly this is a complication arising from the use of Slingshot? It would 
take a 2 maybe hours to completely tear Slingshot out of this app, but I 
would do that if I thought I would then be able to see the error messages. 


On Sunday, October 8, 2017 at 11:46:26 PM UTC-4, tbc++ wrote:
>
> I don't think you can catch an Object on the JVM can you? What happens if 
> you catch Throwable?
>
> On Sun, Oct 8, 2017 at 9:43 PM, > 
> wrote:
>
>> So, for instance, this says that 10 documents were retried:
>>
>> {"message" {:num-slabs 1, :num-active-slabs 1, :enqueued 389, :retried 
>> 10, :completed 378, :in-progress 1}}
>>
>> The only place that I call retry! is in this catch block (this function 
>> runs in a background thread):
>>
>> (defn advance
>>   [message]
>>   {:pre [
>>  (= (type message) durable_queue.Task)
>>  ]}
>>   (slingshot/try+
>>(doseq [[k v] @message]
>>  (when-not (or
>> (= :how-many-rows-and-fields-from-database k)
>> (= :database-table k)
>> (= :denormalized-id k)
>> (= :topic k))
>>(transform
>> (get @message :denormalized-id :missing-denormalized-id)
>> k
>> v
>> (get @message :topic :missing-topic)
>> (get @message :how-many-rows-and-fields-from-database 
>> :missing-how-many-rows-and-fields-from-database
>>
>>(durable/complete! message)
>>(catch Object o
>>  (durable/retry! message)
>>  (slingshot/throw+ {
>> :type ::advance
>> :error o
>> }
>>
>>
>> So clearly, the code has a problem with some of the documents, and so an 
>> error is thrown, and it is caught by the catch block. I would love to see 
>> the error. I can not. If I put println in the catch block, the message 
>> never shows up in the terminal. Likewise if I use Timbre. 
>>
>> I would love to see the error object. My inability to see the error 
>> objects has crippled my productivity. 
>>
>> Can anyone tell me what I need to do to see these errors? 
>>
>>
>>
>>
>>
>> On Sunday, October 8, 2017 at 11:13:10 PM UTC-4, lawrence...@gmail.com 
>> wrote:
>>
>>> I'm trying to use a Zach Tellman's durable-queue to pass information 
>>> from one part of my app to another. I have 3 queues. I also have a function 
>>> that runs every 30 seconds, that is suppose to report some statistics for 
>>> me: 
>>>
>>>   (fn []
>>> (future
>>>   (slingshot/try+
>>>(errors/log "Resource usage: "
>>>   
>>>  (monitoring/show-stats-regarding-resources-used-by-this-app))
>>>(errors/log "Stats about from-mysql-to-tables-queue: 
>>> " (durable/stats from-mysql-to-tables-queue))
>>>(errors/log "Stats about from-tables-to-topics-queue: 
>>> " (durable/stats from-tables-to-topics-queue))
>>>(errors/log "Stats about the 
>>> from-topics-to-persistence-queue: " (durable/stats 
>>> from-topics-to-persistence-queue))
>>>(catch Object o
>>>  (slingshot/throw+ {
>>> :type 
>>> ::cycle-resource-usage-report
>>> :error o
>>> }
>>>)
>>>
>>> I think there is an error here, because I see the first (errors/log) but 
>>> typically not anything after that. Can I rely on the fact that this should 
>>> catch anything: 
>>>
>>>(catch Object o
>>>
>>> Because it doesn't seem to work. Instead, the app runs for 8 minutes, 
>>> then prints data about the queues just once, then never again. Assuming an 
>>> error is occuring, why can't I see the error? 
>>>
>>>
>>> {:timestamp "17-10-09 01:27:45", :level :trace, :hostname 
>>> "ip-172-31-4-54", :message ("STARTING")}
>>>
>>> Loading class `com.mysql.jdbc.Driver'. This is deprecated. The new 
>>> driver class is `com.mysql.cj.jdbc.Driver'. The driver is automatically 
>>> registered via the SPI and manual loading of the driver class is generally 
>>> unnecessary.
>>>
>>>
>>> {:timestamp "17-10-09 01:28:16", :level :trace, :hostname 
>>> "ip-172-31-4-54", :message ("\n\n\n\n\nResource usage: " "Memory in use 
>>> (percentage/used/max-heap): (\"67%\" \"2414M\" \"3568M\")\n\nCPU usage 
>>> (how-many-cpu's/load-average):  [4 1.56]\n\nFree memory in jvm: 
>>> [1201758848]")}
>>>
>>>
>>> {:timestamp "17-10-09 01:28:49", :level :trace, :hostname 
>>> "ip-172-31-4-54", :message ("\n\n\n\n\nResource usage: " "Memory in use 
>>> (percentage/used/max-heap): (\"81%\" \"2900M\" \"3568M\")\n\nCPU usage 
>>> (how-many-cpu's/load-average):  [4 2.64]\n\nFree memory in jvm: 
>>> [699798216]")}
>>>
>>> ;; removing redundant entries
>>>
>>> {:timestamp "17-10-09 01:34:48", :level :trace, :hostname "UnknownHost", 
>>>

Re: Can slingshot/try+ and then catch Object really catch any error?

[lawrence . krubner]

It's a bit absurd, but at the end of core/main I'm now catching Exceptions, 
Throwable and Object. But this doesn't seem to help me see what is 
happening on background threads.


(defn -main [& args]

  (slingshot/try+

   ;; 2017-10-04 -- see this:
   ;; 
https://stackoverflow.com/questions/28908835/ssl-peer-shut-down-incorrectly-in-java
   (System/setProperty "https.protocols" "TLSv1.1")
   
   (.addShutdownHook (Runtime/getRuntime)
 (Thread.
  #(do (println "addShutdownHook triggered. This app is 
shutting down.")
   (stop

   ;; 2017-07-12 -- stealing code from here:
   ;; https://stuartsierra.com/2015/05/27/clojure-uncaught-exceptions
   ;; Assuming require [clojure.tools.logging :as log]
   (Thread/setDefaultUncaughtExceptionHandler
(reify Thread$UncaughtExceptionHandler
  (uncaughtException [_ thread ex]
(println ex "Uncaught background exception on thread " (.getName 
thread) " Caught by Thread/setDefaultUncaughtExceptionHandler in 
core/main.")
(throw (ex-info "/setDefaultUncaughtExceptionHandler. The app has 
died!")

   (start)
   (catch Exception e
 (println e)
 (stack/parse-exception e)
 (throw (ex-info "The app has died!")))
   (catch Throwable t
 (println t)
 (throw (ex-info "The app has died!")))
   (catch Object o
 (println o)
 (throw (ex-info "The app has died!")


On Sunday, October 8, 2017 at 11:46:26 PM UTC-4, tbc++ wrote:
>
> I don't think you can catch an Object on the JVM can you? What happens if 
> you catch Throwable?
>
> On Sun, Oct 8, 2017 at 9:43 PM, > 
> wrote:
>
>> So, for instance, this says that 10 documents were retried:
>>
>> {"message" {:num-slabs 1, :num-active-slabs 1, :enqueued 389, :retried 
>> 10, :completed 378, :in-progress 1}}
>>
>> The only place that I call retry! is in this catch block (this function 
>> runs in a background thread):
>>
>> (defn advance
>>   [message]
>>   {:pre [
>>  (= (type message) durable_queue.Task)
>>  ]}
>>   (slingshot/try+
>>(doseq [[k v] @message]
>>  (when-not (or
>> (= :how-many-rows-and-fields-from-database k)
>> (= :database-table k)
>> (= :denormalized-id k)
>> (= :topic k))
>>(transform
>> (get @message :denormalized-id :missing-denormalized-id)
>> k
>> v
>> (get @message :topic :missing-topic)
>> (get @message :how-many-rows-and-fields-from-database 
>> :missing-how-many-rows-and-fields-from-database
>>
>>(durable/complete! message)
>>(catch Object o
>>  (durable/retry! message)
>>  (slingshot/throw+ {
>> :type ::advance
>> :error o
>> }
>>
>>
>> So clearly, the code has a problem with some of the documents, and so an 
>> error is thrown, and it is caught by the catch block. I would love to see 
>> the error. I can not. If I put println in the catch block, the message 
>> never shows up in the terminal. Likewise if I use Timbre. 
>>
>> I would love to see the error object. My inability to see the error 
>> objects has crippled my productivity. 
>>
>> Can anyone tell me what I need to do to see these errors? 
>>
>>
>>
>>
>>
>> On Sunday, October 8, 2017 at 11:13:10 PM UTC-4, lawrence...@gmail.com 
>> wrote:
>>
>>> I'm trying to use a Zach Tellman's durable-queue to pass information 
>>> from one part of my app to another. I have 3 queues. I also have a function 
>>> that runs every 30 seconds, that is suppose to report some statistics for 
>>> me: 
>>>
>>>   (fn []
>>> (future
>>>   (slingshot/try+
>>>(errors/log "Resource usage: "
>>>   
>>>  (monitoring/show-stats-regarding-resources-used-by-this-app))
>>>(errors/log "Stats about from-mysql-to-tables-queue: 
>>> " (durable/stats from-mysql-to-tables-queue))
>>>(errors/log "Stats about from-tables-to-topics-queue: 
>>> " (durable/stats from-tables-to-topics-queue))
>>>(errors/log "Stats about the 
>>> from-topics-to-persistence-queue: " (durable/stats 
>>> from-topics-to-persistence-queue))
>>>(catch Object o
>>>  (slingshot/throw+ {
>>> :type 
>>> ::cycle-resource-usage-report
>>> :error o
>>> }
>>>)
>>>
>>> I think there is an error here, because I see the first (errors/log) but 
>>> typically not anything after that. Can I rely on the fact that this should 
>>> catch anything: 
>>>
>>>(catch Object o
>>>
>>> Because it doesn't seem to work. Instead, the app runs for 8 minutes, 
>>> then prints data about the queues just on

Re: Can slingshot/try+ and then catch Object really catch any error?

[lawrence . krubner]

Well, the people behind Slingshot have suggested it as the Slingshot 
version of catch Exception, but I like your idea and I'll try it. 

Having said that, as I said in my last post, the only place I call (retry!) 
is inside the catch block, so the code does end up inside of that catch 
block. But I never see any results of printing or throwing from inside the 
catch block. 


On Sunday, October 8, 2017 at 11:46:26 PM UTC-4, tbc++ wrote:
>
> I don't think you can catch an Object on the JVM can you? What happens if 
> you catch Throwable?
>
> On Sun, Oct 8, 2017 at 9:43 PM, > 
> wrote:
>
>> So, for instance, this says that 10 documents were retried:
>>
>> {"message" {:num-slabs 1, :num-active-slabs 1, :enqueued 389, :retried 
>> 10, :completed 378, :in-progress 1}}
>>
>> The only place that I call retry! is in this catch block (this function 
>> runs in a background thread):
>>
>> (defn advance
>>   [message]
>>   {:pre [
>>  (= (type message) durable_queue.Task)
>>  ]}
>>   (slingshot/try+
>>(doseq [[k v] @message]
>>  (when-not (or
>> (= :how-many-rows-and-fields-from-database k)
>> (= :database-table k)
>> (= :denormalized-id k)
>> (= :topic k))
>>(transform
>> (get @message :denormalized-id :missing-denormalized-id)
>> k
>> v
>> (get @message :topic :missing-topic)
>> (get @message :how-many-rows-and-fields-from-database 
>> :missing-how-many-rows-and-fields-from-database
>>
>>(durable/complete! message)
>>(catch Object o
>>  (durable/retry! message)
>>  (slingshot/throw+ {
>> :type ::advance
>> :error o
>> }
>>
>>
>> So clearly, the code has a problem with some of the documents, and so an 
>> error is thrown, and it is caught by the catch block. I would love to see 
>> the error. I can not. If I put println in the catch block, the message 
>> never shows up in the terminal. Likewise if I use Timbre. 
>>
>> I would love to see the error object. My inability to see the error 
>> objects has crippled my productivity. 
>>
>> Can anyone tell me what I need to do to see these errors? 
>>
>>
>>
>>
>>
>> On Sunday, October 8, 2017 at 11:13:10 PM UTC-4, lawrence...@gmail.com 
>> wrote:
>>
>>> I'm trying to use a Zach Tellman's durable-queue to pass information 
>>> from one part of my app to another. I have 3 queues. I also have a function 
>>> that runs every 30 seconds, that is suppose to report some statistics for 
>>> me: 
>>>
>>>   (fn []
>>> (future
>>>   (slingshot/try+
>>>(errors/log "Resource usage: "
>>>   
>>>  (monitoring/show-stats-regarding-resources-used-by-this-app))
>>>(errors/log "Stats about from-mysql-to-tables-queue: 
>>> " (durable/stats from-mysql-to-tables-queue))
>>>(errors/log "Stats about from-tables-to-topics-queue: 
>>> " (durable/stats from-tables-to-topics-queue))
>>>(errors/log "Stats about the 
>>> from-topics-to-persistence-queue: " (durable/stats 
>>> from-topics-to-persistence-queue))
>>>(catch Object o
>>>  (slingshot/throw+ {
>>> :type 
>>> ::cycle-resource-usage-report
>>> :error o
>>> }
>>>)
>>>
>>> I think there is an error here, because I see the first (errors/log) but 
>>> typically not anything after that. Can I rely on the fact that this should 
>>> catch anything: 
>>>
>>>(catch Object o
>>>
>>> Because it doesn't seem to work. Instead, the app runs for 8 minutes, 
>>> then prints data about the queues just once, then never again. Assuming an 
>>> error is occuring, why can't I see the error? 
>>>
>>>
>>> {:timestamp "17-10-09 01:27:45", :level :trace, :hostname 
>>> "ip-172-31-4-54", :message ("STARTING")}
>>>
>>> Loading class `com.mysql.jdbc.Driver'. This is deprecated. The new 
>>> driver class is `com.mysql.cj.jdbc.Driver'. The driver is automatically 
>>> registered via the SPI and manual loading of the driver class is generally 
>>> unnecessary.
>>>
>>>
>>> {:timestamp "17-10-09 01:28:16", :level :trace, :hostname 
>>> "ip-172-31-4-54", :message ("\n\n\n\n\nResource usage: " "Memory in use 
>>> (percentage/used/max-heap): (\"67%\" \"2414M\" \"3568M\")\n\nCPU usage 
>>> (how-many-cpu's/load-average):  [4 1.56]\n\nFree memory in jvm: 
>>> [1201758848]")}
>>>
>>>
>>> {:timestamp "17-10-09 01:28:49", :level :trace, :hostname 
>>> "ip-172-31-4-54", :message ("\n\n\n\n\nResource usage: " "Memory in use 
>>> (percentage/used/max-heap): (\"81%\" \"2900M\" \"3568M\")\n\nCPU usage 
>>> (how-many-cpu's/load-average):  [4 2.64]\n\nFree memory 

Re: Can slingshot/try+ and then catch Object really catch any error?

[lawrence . krubner]

So, for instance, this says that 10 documents were retried:

{"message" {:num-slabs 1, :num-active-slabs 1, :enqueued 389, :retried 10, 
:completed 378, :in-progress 1}}

The only place that I call retry! is in this catch block (this function 
runs in a background thread):

(defn advance
  [message]
  {:pre [
 (= (type message) durable_queue.Task)
 ]}
  (slingshot/try+
   (doseq [[k v] @message]
 (when-not (or
(= :how-many-rows-and-fields-from-database k)
(= :database-table k)
(= :denormalized-id k)
(= :topic k))
   (transform
(get @message :denormalized-id :missing-denormalized-id)
k
v
(get @message :topic :missing-topic)
(get @message :how-many-rows-and-fields-from-database 
:missing-how-many-rows-and-fields-from-database
   
   (durable/complete! message)
   (catch Object o
 (durable/retry! message)
 (slingshot/throw+ {
:type ::advance
:error o
}
   

So clearly, the code has a problem with some of the documents, and so an 
error is thrown, and it is caught by the catch block. I would love to see 
the error. I can not. If I put println in the catch block, the message 
never shows up in the terminal. Likewise if I use Timbre. 

I would love to see the error object. My inability to see the error objects 
has crippled my productivity. 

Can anyone tell me what I need to do to see these errors? 





On Sunday, October 8, 2017 at 11:13:10 PM UTC-4, lawrence...@gmail.com 
wrote:
>
> I'm trying to use a Zach Tellman's durable-queue to pass information from 
> one part of my app to another. I have 3 queues. I also have a function that 
> runs every 30 seconds, that is suppose to report some statistics for me: 
>
>   (fn []
> (future
>   (slingshot/try+
>(errors/log "Resource usage: "
>   
>  (monitoring/show-stats-regarding-resources-used-by-this-app))
>(errors/log "Stats about from-mysql-to-tables-queue: " 
> (durable/stats from-mysql-to-tables-queue))
>(errors/log "Stats about from-tables-to-topics-queue: " 
> (durable/stats from-tables-to-topics-queue))
>(errors/log "Stats about the 
> from-topics-to-persistence-queue: " (durable/stats 
> from-topics-to-persistence-queue))
>(catch Object o
>  (slingshot/throw+ {
> :type ::cycle-resource-usage-report
> :error o
> }
>)
>
> I think there is an error here, because I see the first (errors/log) but 
> typically not anything after that. Can I rely on the fact that this should 
> catch anything: 
>
>(catch Object o
>
> Because it doesn't seem to work. Instead, the app runs for 8 minutes, then 
> prints data about the queues just once, then never again. Assuming an error 
> is occuring, why can't I see the error? 
>
>
> {:timestamp "17-10-09 01:27:45", :level :trace, :hostname 
> "ip-172-31-4-54", :message ("STARTING")}
>
> Loading class `com.mysql.jdbc.Driver'. This is deprecated. The new driver 
> class is `com.mysql.cj.jdbc.Driver'. The driver is automatically registered 
> via the SPI and manual loading of the driver class is generally unnecessary.
>
>
> {:timestamp "17-10-09 01:28:16", :level :trace, :hostname 
> "ip-172-31-4-54", :message ("\n\n\n\n\nResource usage: " "Memory in use 
> (percentage/used/max-heap): (\"67%\" \"2414M\" \"3568M\")\n\nCPU usage 
> (how-many-cpu's/load-average):  [4 1.56]\n\nFree memory in jvm: 
> [1201758848]")}
>
>
> {:timestamp "17-10-09 01:28:49", :level :trace, :hostname 
> "ip-172-31-4-54", :message ("\n\n\n\n\nResource usage: " "Memory in use 
> (percentage/used/max-heap): (\"81%\" \"2900M\" \"3568M\")\n\nCPU usage 
> (how-many-cpu's/load-average):  [4 2.64]\n\nFree memory in jvm: 
> [699798216]")}
>
> ;; removing redundant entries
>
> {:timestamp "17-10-09 01:34:48", :level :trace, :hostname "UnknownHost", 
> :message ("\n\n\n\n\nResource usage: " "Memory in use 
> (percentage/used/max-heap): (\"87%\" \"3118M\" \"3568M\")\n\nCPU usage 
> (how-many-cpu's/load-average):  [4 3.77]\n\nFree memory in jvm: 
> [471249616]")}
>
>
> {:timestamp "17-10-09 01:35:17", :level :trace, :hostname 
> "ip-172-31-4-54", :message ("\n\n\n\n\nResource usage: " "Memory in use 
> (percentage/used/max-heap): (\"87%\" \"3120M\" \"3568M\")\n\nCPU usage 
> (how-many-cpu's/load-average):  [4 3.79]\n\nFree memory in jvm: 
> [468602192]")}
>
>
> {:timestamp "17-10-09 01:35:35", :level :trace, :hostname 
> "ip-172-31-4-54", :message ("\nStats about from-mysql-to-tables-queue: " 
> {"message" {:num-slabs 1, :num-active-slabs 1, :enqueued 306671, :retrie

Re: Can slingshot/try+ and then catch Object really catch any error?

[lawrence . krubner]

Using spit to an append to a log file, I end up seeing much more than what 
I can see from Timbre. The app writes a few lines, then dies: 

("\n\n\n\n\nResource usage: " "Memory in use (percentage/used/max-heap): 
(\"3%\" \"133M\" \"3568M\")\n\nCPU usage (how-many-cpu's/load-average):  [4 
0.0]\n\nFree memory in jvm: [252719920]") 

("\nStats about from-mysql-to-tables-queue: " {"message" {:num-slabs 1, 
:num-active-slabs 1, :enqueued 389, :retried 10, :completed 378, 
:in-progress 1}}) 

("\nStats about from-tables-to-topics-queue: " {}) 

("\nStats about the from-topics-to-persistence-queue: " {}) 

("\n\n\n\n\nResource usage: " "Memory in use (percentage/used/max-heap): 
(\"3%\" \"134M\" \"3568M\")\n\nCPU usage (how-many-cpu's/load-average):  [4 
0.0]\n\nFree memory in jvm: [251769008]") 

("\nStats about from-mysql-to-tables-queue: " {"message" {:num-slabs 1, 
:num-active-slabs 1, :enqueued 389, :retried 10, :completed 378, 
:in-progress 1}}) 

("\nStats about from-tables-to-topics-queue: " {}) 

("\nStats about the from-topics-to-persistence-queue: " {}) 

Still, I feel like I've got everything wrapped in slingshot/try+ and catch 
and throw, yet I don't see the error that is killing the app. Is that 
because of (catch Object) doesn't catch all the errors that I should be 
checking for, or is that because my logging functions just don't write 
them? 


On Sunday, October 8, 2017 at 11:13:10 PM UTC-4, lawrence...@gmail.com 
wrote:
>
> I'm trying to use a Zach Tellman's durable-queue to pass information from 
> one part of my app to another. I have 3 queues. I also have a function that 
> runs every 30 seconds, that is suppose to report some statistics for me: 
>
>   (fn []
> (future
>   (slingshot/try+
>(errors/log "Resource usage: "
>   
>  (monitoring/show-stats-regarding-resources-used-by-this-app))
>(errors/log "Stats about from-mysql-to-tables-queue: " 
> (durable/stats from-mysql-to-tables-queue))
>(errors/log "Stats about from-tables-to-topics-queue: " 
> (durable/stats from-tables-to-topics-queue))
>(errors/log "Stats about the 
> from-topics-to-persistence-queue: " (durable/stats 
> from-topics-to-persistence-queue))
>(catch Object o
>  (slingshot/throw+ {
> :type ::cycle-resource-usage-report
> :error o
> }
>)
>
> I think there is an error here, because I see the first (errors/log) but 
> typically not anything after that. Can I rely on the fact that this should 
> catch anything: 
>
>(catch Object o
>
> Because it doesn't seem to work. Instead, the app runs for 8 minutes, then 
> prints data about the queues just once, then never again. Assuming an error 
> is occuring, why can't I see the error? 
>
>
> {:timestamp "17-10-09 01:27:45", :level :trace, :hostname 
> "ip-172-31-4-54", :message ("STARTING")}
>
> Loading class `com.mysql.jdbc.Driver'. This is deprecated. The new driver 
> class is `com.mysql.cj.jdbc.Driver'. The driver is automatically registered 
> via the SPI and manual loading of the driver class is generally unnecessary.
>
>
> {:timestamp "17-10-09 01:28:16", :level :trace, :hostname 
> "ip-172-31-4-54", :message ("\n\n\n\n\nResource usage: " "Memory in use 
> (percentage/used/max-heap): (\"67%\" \"2414M\" \"3568M\")\n\nCPU usage 
> (how-many-cpu's/load-average):  [4 1.56]\n\nFree memory in jvm: 
> [1201758848]")}
>
>
> {:timestamp "17-10-09 01:28:49", :level :trace, :hostname 
> "ip-172-31-4-54", :message ("\n\n\n\n\nResource usage: " "Memory in use 
> (percentage/used/max-heap): (\"81%\" \"2900M\" \"3568M\")\n\nCPU usage 
> (how-many-cpu's/load-average):  [4 2.64]\n\nFree memory in jvm: 
> [699798216]")}
>
> ;; removing redundant entries
>
> {:timestamp "17-10-09 01:34:48", :level :trace, :hostname "UnknownHost", 
> :message ("\n\n\n\n\nResource usage: " "Memory in use 
> (percentage/used/max-heap): (\"87%\" \"3118M\" \"3568M\")\n\nCPU usage 
> (how-many-cpu's/load-average):  [4 3.77]\n\nFree memory in jvm: 
> [471249616]")}
>
>
> {:timestamp "17-10-09 01:35:17", :level :trace, :hostname 
> "ip-172-31-4-54", :message ("\n\n\n\n\nResource usage: " "Memory in use 
> (percentage/used/max-heap): (\"87%\" \"3120M\" \"3568M\")\n\nCPU usage 
> (how-many-cpu's/load-average):  [4 3.79]\n\nFree memory in jvm: 
> [468602192]")}
>
>
> {:timestamp "17-10-09 01:35:35", :level :trace, :hostname 
> "ip-172-31-4-54", :message ("\nStats about from-mysql-to-tables-queue: " 
> {"message" {:num-slabs 1, :num-active-slabs 1, :enqueued 306671, :retried 
> 0, :completed 306660, :in-progress 8}})}
>
>
> {:timestamp "17-10-09 01:35:50", :level :trace, :hostname 
> "ip-172-31-4-54", :message ("\nStats about from-tables-to-topics-

Re: Can slingshot/try+ and then catch Object really catch any error?

[lawrence . krubner]

More generally, I'm having trouble seeing any kind of logging messages. 
Even when I delete this Timbre config, and simply go with Timbre defaults, 
and set the level to "trace", I only erratically see anything printed to 
the logs. I assume there are messages dying in background threads, but I 
have a bit of code, which I stole from a blog post by Stuart Sierra, and 
which I put in my core/main function, which should, I assumed, reveal to me 
errors thrown from background threads. 


On Sunday, October 8, 2017 at 11:13:10 PM UTC-4, lawrence...@gmail.com 
wrote:
>
> I'm trying to use a Zach Tellman's durable-queue to pass information from 
> one part of my app to another. I have 3 queues. I also have a function that 
> runs every 30 seconds, that is suppose to report some statistics for me: 
>
>   (fn []
> (future
>   (slingshot/try+
>(errors/log "Resource usage: "
>   
>  (monitoring/show-stats-regarding-resources-used-by-this-app))
>(errors/log "Stats about from-mysql-to-tables-queue: " 
> (durable/stats from-mysql-to-tables-queue))
>(errors/log "Stats about from-tables-to-topics-queue: " 
> (durable/stats from-tables-to-topics-queue))
>(errors/log "Stats about the 
> from-topics-to-persistence-queue: " (durable/stats 
> from-topics-to-persistence-queue))
>(catch Object o
>  (slingshot/throw+ {
> :type ::cycle-resource-usage-report
> :error o
> }
>)
>
> I think there is an error here, because I see the first (errors/log) but 
> typically not anything after that. Can I rely on the fact that this should 
> catch anything: 
>
>(catch Object o
>
> Because it doesn't seem to work. Instead, the app runs for 8 minutes, then 
> prints data about the queues just once, then never again. Assuming an error 
> is occuring, why can't I see the error? 
>
>
> {:timestamp "17-10-09 01:27:45", :level :trace, :hostname 
> "ip-172-31-4-54", :message ("STARTING")}
>
> Loading class `com.mysql.jdbc.Driver'. This is deprecated. The new driver 
> class is `com.mysql.cj.jdbc.Driver'. The driver is automatically registered 
> via the SPI and manual loading of the driver class is generally unnecessary.
>
>
> {:timestamp "17-10-09 01:28:16", :level :trace, :hostname 
> "ip-172-31-4-54", :message ("\n\n\n\n\nResource usage: " "Memory in use 
> (percentage/used/max-heap): (\"67%\" \"2414M\" \"3568M\")\n\nCPU usage 
> (how-many-cpu's/load-average):  [4 1.56]\n\nFree memory in jvm: 
> [1201758848]")}
>
>
> {:timestamp "17-10-09 01:28:49", :level :trace, :hostname 
> "ip-172-31-4-54", :message ("\n\n\n\n\nResource usage: " "Memory in use 
> (percentage/used/max-heap): (\"81%\" \"2900M\" \"3568M\")\n\nCPU usage 
> (how-many-cpu's/load-average):  [4 2.64]\n\nFree memory in jvm: 
> [699798216]")}
>
> ;; removing redundant entries
>
> {:timestamp "17-10-09 01:34:48", :level :trace, :hostname "UnknownHost", 
> :message ("\n\n\n\n\nResource usage: " "Memory in use 
> (percentage/used/max-heap): (\"87%\" \"3118M\" \"3568M\")\n\nCPU usage 
> (how-many-cpu's/load-average):  [4 3.77]\n\nFree memory in jvm: 
> [471249616]")}
>
>
> {:timestamp "17-10-09 01:35:17", :level :trace, :hostname 
> "ip-172-31-4-54", :message ("\n\n\n\n\nResource usage: " "Memory in use 
> (percentage/used/max-heap): (\"87%\" \"3120M\" \"3568M\")\n\nCPU usage 
> (how-many-cpu's/load-average):  [4 3.79]\n\nFree memory in jvm: 
> [468602192]")}
>
>
> {:timestamp "17-10-09 01:35:35", :level :trace, :hostname 
> "ip-172-31-4-54", :message ("\nStats about from-mysql-to-tables-queue: " 
> {"message" {:num-slabs 1, :num-active-slabs 1, :enqueued 306671, :retried 
> 0, :completed 306660, :in-progress 8}})}
>
>
> {:timestamp "17-10-09 01:35:50", :level :trace, :hostname 
> "ip-172-31-4-54", :message ("\nStats about from-tables-to-topics-queue: " 
> {})}
>
>
> {:timestamp "17-10-09 01:35:50", :level :trace, :hostname 
> "ip-172-31-4-54", :message ("\nStats about the 
> from-topics-to-persistence-queue: " {})}
>
>
> {:timestamp "17-10-09 01:36:15", :level :trace, :hostname 
> "ip-172-31-4-54", :message ("\n\n\n\n\nResource usage: " "Memory in use 
> (percentage/used/max-heap): (\"28%\" \"1007M\" \"3568M\")\n\nCPU usage 
> (how-many-cpu's/load-average):  [4 4.14]\n\nFree memory in jvm: 
> [1868654000]")}
>
>
>
>
>
> So 8 minutes after the app starts, it print the lines with stats about the 
> queues, just this one time, and then never again. I think an error must 
> occur inside of the function that is suppose to report the queues. What 
> would I have to do to see that error?
>
> My core/main function looks like this:
>
> (defn -main [& args]
>
>   (slingshot/try+
>
>;; 2017-10-04 -- see this:
>;; 
> ht

Re: Can slingshot/try+ and then catch Object really catch any error?

[lawrence . krubner]

(errors/log) simply writes to the Timbre logging library. I thought perhaps 
there was an error in my Timbre configuration, so I wrapped those functions 
in try/catch, but I still don't see anything. I'm trying this as my Timbre 
config:




(defn- set-timbre-level []
  (timbre/set-config!
   {:level :trace ; e/o #{:trace :debug :info :warn :error :fatal :report}
:output-fn (fn [data]
 (slingshot/try+
   (clojure.string/join "\n" data)
   (catch Object o
 (slingshot/throw+ {
:type ::output-fn-of-timbre
:error o
}
 
:appenders {
;; Appender id
:for-stdout {
 :enabled?   true
 :async? false
 :min-level  nil
 :rate-limit [[1 250] [10 5000]] ; 1/250ms, 
10/5s
 :output-fn  :inherit
 ;; Appender's fn
 :fn (fn
   [{:keys [vargs_ hostname_ timestamp_ 
level output-fn] :as args}]
   (slingshot/try+
 (let [
   messages (map (fn [msg] { 
:timestamp @timestamp_
:level 
level

:hostname  @hostname_

:message   msg })
 @vargs_)
   ]
   (println (str 
(temporal/current-time-as-string) "\n" (output-fn messages
 (catch Object o
   (slingshot/throw+ {
  :type 
::for-stdout-fn-of-timbre
  :error o
  }
 }}}))

On Sunday, October 8, 2017 at 11:13:10 PM UTC-4, lawrence...@gmail.com 
wrote:
>
> I'm trying to use a Zach Tellman's durable-queue to pass information from 
> one part of my app to another. I have 3 queues. I also have a function that 
> runs every 30 seconds, that is suppose to report some statistics for me: 
>
>   (fn []
> (future
>   (slingshot/try+
>(errors/log "Resource usage: "
>   
>  (monitoring/show-stats-regarding-resources-used-by-this-app))
>(errors/log "Stats about from-mysql-to-tables-queue: " 
> (durable/stats from-mysql-to-tables-queue))
>(errors/log "Stats about from-tables-to-topics-queue: " 
> (durable/stats from-tables-to-topics-queue))
>(errors/log "Stats about the 
> from-topics-to-persistence-queue: " (durable/stats 
> from-topics-to-persistence-queue))
>(catch Object o
>  (slingshot/throw+ {
> :type ::cycle-resource-usage-report
> :error o
> }
>)
>
> I think there is an error here, because I see the first (errors/log) but 
> typically not anything after that. Can I rely on the fact that this should 
> catch anything: 
>
>(catch Object o
>
> Because it doesn't seem to work. Instead, the app runs for 8 minutes, then 
> prints data about the queues just once, then never again. Assuming an error 
> is occuring, why can't I see the error? 
>
>
> {:timestamp "17-10-09 01:27:45", :level :trace, :hostname 
> "ip-172-31-4-54", :message ("STARTING")}
>
> Loading class `com.mysql.jdbc.Driver'. This is deprecated. The new driver 
> class is `com.mysql.cj.jdbc.Driver'. The driver is automatically registered 
> via the SPI and manual loading of the driver class is generally unnecessary.
>
>
> {:timestamp "17-10-09 01:28:16", :level :trace, :hostname 
> "ip-172-31-4-54", :message ("\n\n\n\n\nResource usage: " "Memory in use 
> (percentage/used/max-heap): (\"67%\" \"2414M\" \"3568M\")\n\nCPU usage 
> (how-many-cpu's/load-average):  [4 1.56]\n\nFree memory in jvm: 
> [1201758848]")}
>
>
> {:timestamp "17-10-09 01:28:49", :level :trace, :hostname 
> "ip-172-31-4-54", :message ("\n\n\n\n\nResource usage: " "Memory in use 
> (percentage/used/max-heap): (\"81%\" \"2900M\" \"3568M\")\n\nCPU usage 
> (how-many-cpu's/load-average):  [4 2.64]\n\nFree memory in jvm: 
> [699798216]")}
>
> ;; removing redundant entries
>
> {:timestamp "17

Can slingshot/try+ and then catch Object really catch any error?

[lawrence . krubner]

I'm trying to use a Zach Tellman's durable-queue to pass information from 
one part of my app to another. I have 3 queues. I also have a function that 
runs every 30 seconds, that is suppose to report some statistics for me: 

  (fn []
(future
  (slingshot/try+
   (errors/log "Resource usage: "
  
 (monitoring/show-stats-regarding-resources-used-by-this-app))
   (errors/log "Stats about from-mysql-to-tables-queue: " 
(durable/stats from-mysql-to-tables-queue))
   (errors/log "Stats about from-tables-to-topics-queue: " 
(durable/stats from-tables-to-topics-queue))
   (errors/log "Stats about the 
from-topics-to-persistence-queue: " (durable/stats 
from-topics-to-persistence-queue))
   (catch Object o
 (slingshot/throw+ {
:type ::cycle-resource-usage-report
:error o
}
   )

I think there is an error here, because I see the first (errors/log) but 
typically not anything after that. Can I rely on the fact that this should 
catch anything: 

   (catch Object o

Because it doesn't seem to work. Instead, the app runs for 8 minutes, then 
prints data about the queues just once, then never again. Assuming an error 
is occuring, why can't I see the error? 


{:timestamp "17-10-09 01:27:45", :level :trace, :hostname "ip-172-31-4-54", 
:message ("STARTING")}

Loading class `com.mysql.jdbc.Driver'. This is deprecated. The new driver 
class is `com.mysql.cj.jdbc.Driver'. The driver is automatically registered 
via the SPI and manual loading of the driver class is generally unnecessary.


{:timestamp "17-10-09 01:28:16", :level :trace, :hostname "ip-172-31-4-54", 
:message ("\n\n\n\n\nResource usage: " "Memory in use 
(percentage/used/max-heap): (\"67%\" \"2414M\" \"3568M\")\n\nCPU usage 
(how-many-cpu's/load-average):  [4 1.56]\n\nFree memory in jvm: 
[1201758848]")}


{:timestamp "17-10-09 01:28:49", :level :trace, :hostname "ip-172-31-4-54", 
:message ("\n\n\n\n\nResource usage: " "Memory in use 
(percentage/used/max-heap): (\"81%\" \"2900M\" \"3568M\")\n\nCPU usage 
(how-many-cpu's/load-average):  [4 2.64]\n\nFree memory in jvm: 
[699798216]")}

;; removing redundant entries

{:timestamp "17-10-09 01:34:48", :level :trace, :hostname "UnknownHost", 
:message ("\n\n\n\n\nResource usage: " "Memory in use 
(percentage/used/max-heap): (\"87%\" \"3118M\" \"3568M\")\n\nCPU usage 
(how-many-cpu's/load-average):  [4 3.77]\n\nFree memory in jvm: 
[471249616]")}


{:timestamp "17-10-09 01:35:17", :level :trace, :hostname "ip-172-31-4-54", 
:message ("\n\n\n\n\nResource usage: " "Memory in use 
(percentage/used/max-heap): (\"87%\" \"3120M\" \"3568M\")\n\nCPU usage 
(how-many-cpu's/load-average):  [4 3.79]\n\nFree memory in jvm: 
[468602192]")}


{:timestamp "17-10-09 01:35:35", :level :trace, :hostname "ip-172-31-4-54", 
:message ("\nStats about from-mysql-to-tables-queue: " {"message" 
{:num-slabs 1, :num-active-slabs 1, :enqueued 306671, :retried 0, 
:completed 306660, :in-progress 8}})}


{:timestamp "17-10-09 01:35:50", :level :trace, :hostname "ip-172-31-4-54", 
:message ("\nStats about from-tables-to-topics-queue: " {})}


{:timestamp "17-10-09 01:35:50", :level :trace, :hostname "ip-172-31-4-54", 
:message ("\nStats about the from-topics-to-persistence-queue: " {})}


{:timestamp "17-10-09 01:36:15", :level :trace, :hostname "ip-172-31-4-54", 
:message ("\n\n\n\n\nResource usage: " "Memory in use 
(percentage/used/max-heap): (\"28%\" \"1007M\" \"3568M\")\n\nCPU usage 
(how-many-cpu's/load-average):  [4 4.14]\n\nFree memory in jvm: 
[1868654000]")}





So 8 minutes after the app starts, it print the lines with stats about the 
queues, just this one time, and then never again. I think an error must 
occur inside of the function that is suppose to report the queues. What 
would I have to do to see that error?

My core/main function looks like this:

(defn -main [& args]

  (slingshot/try+

   ;; 2017-10-04 -- see this:
   ;; 
https://stackoverflow.com/questions/28908835/ssl-peer-shut-down-incorrectly-in-java
   (System/setProperty "https.protocols" "TLSv1.1")
   
   (.addShutdownHook (Runtime/getRuntime)
 (Thread.
  #(do (println "addShutdownHook triggered. This app is 
shutting down.")
   (stop

   ;; 2017-07-12 -- stealing code from here:
   ;; https://stuartsierra.com/2015/05/27/clojure-uncaught-exceptions
   ;; Assuming require [clojure.tools.logging :as log]
   (Thread/setDefaultUncaughtExceptionHandler
(reify Thread$UncaughtExceptionHandler
  (uncaughtException [_ thread ex]
(println ex "Uncaught background exception on thread " (.getName 
thread) " Caught by Thread/setDefaultUncaughtExcept

Re: How to get an error message from Elastisch?

[lawrence . krubner]

I did find this:

http://docs.aws.amazon.com/AWSEC2/latest/UserGuide/EBSPerformance.html

"Your performance can also be impacted if your application isn’t sending 
enough I/O requests. This can be monitored by looking at your volume’s 
queue length and I/O size. The queue length is the number of pending I/O 
requests from your application to your volume. For maximum consistency, 
HDD-backed volumes must maintain a queue length (rounded to the nearest 
whole number) of 4 or more when performing 1 MiB sequential I/O. "

So, at least for now, I'll assume I'm dealing with some weird kind of AWS 
thing. But if anyone has seen this pattern (slow until you get a Socket 
timeout, then fast) because of something you did in your Clojure code, I 
would love to hear about it. 




On Thursday, October 5, 2017 at 2:31:10 PM UTC-4, lawrence...@gmail.com 
wrote:
>
> Konstantinov, yes, perhaps, though I can't think where. This is a small 
> app, about 950 lines of code. There is a limited number of places where I 
> can make such a mistake. I do aggregate a lot of data into an atom, and I'm 
> sure there is a lot of contention around the atom, but the Socket timeout 
> errors are surely some weird AWS or ElasticSearch thing. 
>
>
>
> On Thursday, October 5, 2017 at 2:12:42 PM UTC-4, Lubomir Konstantinov 
> wrote:
>>
>> Holding to a head somewhere perhaps?
>>
>> On Thursday, 5 October 2017 19:32:11 UTC+3, lawrence...@gmail.com wrote:
>>>
>>> One last thing, I should mention, if I go through and add " LIMIT 100 " 
>>> to all the SQL queries, everything works great. That is, when dealing with 
>>> a few hundred documents, the app seems to work perfectly, and there are no 
>>> errors. It's only when I try to work with a few million documents that 
>>> things fall apart. I suspect some kind of contention arises... somewhere. 
>>> Possibly in several locations. But I'm not yet sure what the problem is. 
>>>
>>>
>>>
>>> On Wednesday, October 4, 2017 at 12:49:12 AM UTC-4, 
>>> lawrence...@gmail.com wrote:

 This is probably a stupid question, but is there an obvious way to get 
 an error message out of Elastisch? I had an app that was working with 
 MongoDB and I was then told I had to use ElasticSearch instead (something 
 about only using AWS for everything) so now I'm trying to get an error 
 message, because my code doesn't seem to work. I read through here without 
 seeing anything obvious: 

 http://clojureelasticsearch.info/articles/getting_started.htm

 I rewrote my MongoDB function, so it should work with Elastisch: 

 (defn push-item-to-persistence
   [item db]
   (let [
 denormalized-id (get-in item [:denormalized-id] :no-id)
 item (assoc item :updated-at 
 (temporal/current-time-as-datetime))
 item (assoc item :permanent-holding-id-for-item-instances 
 (java.util.UUID/randomUUID))
 item (assoc item :instance-id-for-this-one-item 
 (java.util.UUID/randomUUID))
 item (assoc item :item-type :deduplication)
 ]
 (if (= denormalized-id :no-id)
   (slingshot/throw+ {
  :type 
 ::no-denormalized-id-in-push-item-into-database
  :item item
  })
   (slingshot/try+
(put conn index mapping-type id document)
(esd/put db "facts-over-time" "deduplicaton" (str 
 denormalized-id) item)
(println " done with put in push-item-to-persistence ")
(catch Object o
  (slingshot/throw+ {
 :type ::push-item-to-persistence
 :error o
 :item item
 :db db
 }
))


 It doesn't seem that any documents are getting into ElasticSearch. I 
 was hoping to the (throw) would reveal to me some useful debugging 
 information, but that doesn't seem to happen. 

 The connection appears to be valid, as I can do this:

  (let [conn (persistence/multi-thread-start config)
res  (esd/search conn "facts-over-time" 
 "deduplication" :query {:match_all {}})
n(esrsp/total-hits res)
hits (esrsp/hits-from res)]
(clojure.pprint/pprint res))

 and I get: 

 {:took 1,
  :timed_out false,
  :_shards {:total 5, :successful 5, :failed 0},
  :hits {:total 0, :max_score nil, :hits []}}

 So the connection is there. But no records are. 




-- 
You received this message because you are subscribed to the Google
Groups "Clojure" group.
To post to this group, send email to clojure@googlegroups.com
Note that posts from new members are moderated - please be patient with 

Re: How to get an error message from Elastisch?

[lawrence . krubner]

Konstantinov, yes, perhaps, though I can't think where. This is a small 
app, about 950 lines of code. There is a limited number of places where I 
can make such a mistake. I do aggregate a lot of data into an atom, and I'm 
sure there is a lot of contention around the atom, but the Socket timeout 
errors are surely some weird AWS or ElasticSearch thing. 



On Thursday, October 5, 2017 at 2:12:42 PM UTC-4, Lubomir Konstantinov 
wrote:
>
> Holding to a head somewhere perhaps?
>
> On Thursday, 5 October 2017 19:32:11 UTC+3, lawrence...@gmail.com wrote:
>>
>> One last thing, I should mention, if I go through and add " LIMIT 100 " 
>> to all the SQL queries, everything works great. That is, when dealing with 
>> a few hundred documents, the app seems to work perfectly, and there are no 
>> errors. It's only when I try to work with a few million documents that 
>> things fall apart. I suspect some kind of contention arises... somewhere. 
>> Possibly in several locations. But I'm not yet sure what the problem is. 
>>
>>
>>
>> On Wednesday, October 4, 2017 at 12:49:12 AM UTC-4, lawrence...@gmail.com 
>> wrote:
>>>
>>> This is probably a stupid question, but is there an obvious way to get 
>>> an error message out of Elastisch? I had an app that was working with 
>>> MongoDB and I was then told I had to use ElasticSearch instead (something 
>>> about only using AWS for everything) so now I'm trying to get an error 
>>> message, because my code doesn't seem to work. I read through here without 
>>> seeing anything obvious: 
>>>
>>> http://clojureelasticsearch.info/articles/getting_started.htm
>>>
>>> I rewrote my MongoDB function, so it should work with Elastisch: 
>>>
>>> (defn push-item-to-persistence
>>>   [item db]
>>>   (let [
>>> denormalized-id (get-in item [:denormalized-id] :no-id)
>>> item (assoc item :updated-at (temporal/current-time-as-datetime))
>>> item (assoc item :permanent-holding-id-for-item-instances 
>>> (java.util.UUID/randomUUID))
>>> item (assoc item :instance-id-for-this-one-item 
>>> (java.util.UUID/randomUUID))
>>> item (assoc item :item-type :deduplication)
>>> ]
>>> (if (= denormalized-id :no-id)
>>>   (slingshot/throw+ {
>>>  :type 
>>> ::no-denormalized-id-in-push-item-into-database
>>>  :item item
>>>  })
>>>   (slingshot/try+
>>>(put conn index mapping-type id document)
>>>(esd/put db "facts-over-time" "deduplicaton" (str 
>>> denormalized-id) item)
>>>(println " done with put in push-item-to-persistence ")
>>>(catch Object o
>>>  (slingshot/throw+ {
>>> :type ::push-item-to-persistence
>>> :error o
>>> :item item
>>> :db db
>>> }
>>>))
>>>
>>>
>>> It doesn't seem that any documents are getting into ElasticSearch. I was 
>>> hoping to the (throw) would reveal to me some useful debugging information, 
>>> but that doesn't seem to happen. 
>>>
>>> The connection appears to be valid, as I can do this:
>>>
>>>  (let [conn (persistence/multi-thread-start config)
>>>res  (esd/search conn "facts-over-time" 
>>> "deduplication" :query {:match_all {}})
>>>n(esrsp/total-hits res)
>>>hits (esrsp/hits-from res)]
>>>(clojure.pprint/pprint res))
>>>
>>> and I get: 
>>>
>>> {:took 1,
>>>  :timed_out false,
>>>  :_shards {:total 5, :successful 5, :failed 0},
>>>  :hits {:total 0, :max_score nil, :hits []}}
>>>
>>> So the connection is there. But no records are. 
>>>
>>>
>>>

-- 
You received this message because you are subscribed to the Google
Groups "Clojure" group.
To post to this group, send email to clojure@googlegroups.com
Note that posts from new members are moderated - please be patient with your 
first post.
To unsubscribe from this group, send email to
clojure+unsubscr...@googlegroups.com
For more options, visit this group at
http://groups.google.com/group/clojure?hl=en
--- 
You received this message because you are subscribed to the Google Groups 
"Clojure" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to clojure+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: How to get an error message from Elastisch?

[lawrence . krubner]

Obviously I'm brain-dead, since I forgot to retry the write on failure. So 
I fixed this now:
 
(defn advance
  [message db]
  {:pre [
 (= (type message) durable_queue.Task)
 ]}
  (let [
;; 2017-10-05 -- if this is successful, then the return will look 
like this:
;; {:_index facts-over-time, :_type deduplicaton, :_id 
investor-6757, :_version 1, :result created, :_shards {:total 2, 
:successful 1, :failed 0}, :created true}
success-or-failure (persistence/push-item-to-persistence @message 
db)
]
(if (= (get-in success-or-failure [:_shards :successful]) 1) 
  (durable/complete! message)
  (durable/retry! message))

But my question remains: why would the app crawl at a glacial speed, till I 
get those SocketConnection errors, and then it goes fast? 

I'm not sure if this is a Clojure question or an ElasticSearch AWS 
question. I'm also researching what sort of rate limits AWS imposes, so I 
can figure out why I would get SocketConnection errors at first. 

If I got zero writes at first, then I could believe that all 20 of my 
threads were blocking, and then they all timeout, then they all start going 
fast. I know AWS load balancing has an algorithm that only slowly adapts to 
spikes in demand. I have trouble believing they would apply that kind of 
load balancing to their ElasticSearch service, but that would explain what 
I'm seeing. I have not been able to find a single article online that 
suggests this. 




On Thursday, October 5, 2017 at 12:32:11 PM UTC-4, lawrence...@gmail.com 
wrote:
>
> One last thing, I should mention, if I go through and add " LIMIT 100 " to 
> all the SQL queries, everything works great. That is, when dealing with a 
> few hundred documents, the app seems to work perfectly, and there are no 
> errors. It's only when I try to work with a few million documents that 
> things fall apart. I suspect some kind of contention arises... somewhere. 
> Possibly in several locations. But I'm not yet sure what the problem is. 
>
>
>
> On Wednesday, October 4, 2017 at 12:49:12 AM UTC-4, lawrence...@gmail.com 
> wrote:
>>
>> This is probably a stupid question, but is there an obvious way to get an 
>> error message out of Elastisch? I had an app that was working with MongoDB 
>> and I was then told I had to use ElasticSearch instead (something about 
>> only using AWS for everything) so now I'm trying to get an error message, 
>> because my code doesn't seem to work. I read through here without seeing 
>> anything obvious: 
>>
>> http://clojureelasticsearch.info/articles/getting_started.htm
>>
>> I rewrote my MongoDB function, so it should work with Elastisch: 
>>
>> (defn push-item-to-persistence
>>   [item db]
>>   (let [
>> denormalized-id (get-in item [:denormalized-id] :no-id)
>> item (assoc item :updated-at (temporal/current-time-as-datetime))
>> item (assoc item :permanent-holding-id-for-item-instances 
>> (java.util.UUID/randomUUID))
>> item (assoc item :instance-id-for-this-one-item 
>> (java.util.UUID/randomUUID))
>> item (assoc item :item-type :deduplication)
>> ]
>> (if (= denormalized-id :no-id)
>>   (slingshot/throw+ {
>>  :type 
>> ::no-denormalized-id-in-push-item-into-database
>>  :item item
>>  })
>>   (slingshot/try+
>>(put conn index mapping-type id document)
>>(esd/put db "facts-over-time" "deduplicaton" (str denormalized-id) 
>> item)
>>(println " done with put in push-item-to-persistence ")
>>(catch Object o
>>  (slingshot/throw+ {
>> :type ::push-item-to-persistence
>> :error o
>> :item item
>> :db db
>> }
>>))
>>
>>
>> It doesn't seem that any documents are getting into ElasticSearch. I was 
>> hoping to the (throw) would reveal to me some useful debugging information, 
>> but that doesn't seem to happen. 
>>
>> The connection appears to be valid, as I can do this:
>>
>>  (let [conn (persistence/multi-thread-start config)
>>res  (esd/search conn "facts-over-time" 
>> "deduplication" :query {:match_all {}})
>>n(esrsp/total-hits res)
>>hits (esrsp/hits-from res)]
>>(clojure.pprint/pprint res))
>>
>> and I get: 
>>
>> {:took 1,
>>  :timed_out false,
>>  :_shards {:total 5, :successful 5, :failed 0},
>>  :hits {:total 0, :max_score nil, :hits []}}
>>
>> So the connection is there. But no records are. 
>>
>>
>>

-- 
You received this message because you are subscribed to the Google
Groups "Clojure" group.
To post to this group, send email to clojure@googlegroups.com
Note that posts from new members are moderated - please be patient with your 
first post.
To unsubscri

Re: How to get an error message from Elastisch?

[lawrence . krubner]

One last thing, I should mention, if I go through and add " LIMIT 100 " to 
all the SQL queries, everything works great. That is, when dealing with a 
few hundred documents, the app seems to work perfectly, and there are no 
errors. It's only when I try to work with a few million documents that 
things fall apart. I suspect some kind of contention arises... somewhere. 
Possibly in several locations. But I'm not yet sure what the problem is. 



On Wednesday, October 4, 2017 at 12:49:12 AM UTC-4, lawrence...@gmail.com 
wrote:
>
> This is probably a stupid question, but is there an obvious way to get an 
> error message out of Elastisch? I had an app that was working with MongoDB 
> and I was then told I had to use ElasticSearch instead (something about 
> only using AWS for everything) so now I'm trying to get an error message, 
> because my code doesn't seem to work. I read through here without seeing 
> anything obvious: 
>
> http://clojureelasticsearch.info/articles/getting_started.htm
>
> I rewrote my MongoDB function, so it should work with Elastisch: 
>
> (defn push-item-to-persistence
>   [item db]
>   (let [
> denormalized-id (get-in item [:denormalized-id] :no-id)
> item (assoc item :updated-at (temporal/current-time-as-datetime))
> item (assoc item :permanent-holding-id-for-item-instances 
> (java.util.UUID/randomUUID))
> item (assoc item :instance-id-for-this-one-item 
> (java.util.UUID/randomUUID))
> item (assoc item :item-type :deduplication)
> ]
> (if (= denormalized-id :no-id)
>   (slingshot/throw+ {
>  :type 
> ::no-denormalized-id-in-push-item-into-database
>  :item item
>  })
>   (slingshot/try+
>(put conn index mapping-type id document)
>(esd/put db "facts-over-time" "deduplicaton" (str denormalized-id) 
> item)
>(println " done with put in push-item-to-persistence ")
>(catch Object o
>  (slingshot/throw+ {
> :type ::push-item-to-persistence
> :error o
> :item item
> :db db
> }
>))
>
>
> It doesn't seem that any documents are getting into ElasticSearch. I was 
> hoping to the (throw) would reveal to me some useful debugging information, 
> but that doesn't seem to happen. 
>
> The connection appears to be valid, as I can do this:
>
>  (let [conn (persistence/multi-thread-start config)
>res  (esd/search conn "facts-over-time" 
> "deduplication" :query {:match_all {}})
>n(esrsp/total-hits res)
>hits (esrsp/hits-from res)]
>(clojure.pprint/pprint res))
>
> and I get: 
>
> {:took 1,
>  :timed_out false,
>  :_shards {:total 5, :successful 5, :failed 0},
>  :hits {:total 0, :max_score nil, :hits []}}
>
> So the connection is there. But no records are. 
>
>
>

-- 
You received this message because you are subscribed to the Google
Groups "Clojure" group.
To post to this group, send email to clojure@googlegroups.com
Note that posts from new members are moderated - please be patient with your 
first post.
To unsubscribe from this group, send email to
clojure+unsubscr...@googlegroups.com
For more options, visit this group at
http://groups.google.com/group/clojure?hl=en
--- 
You received this message because you are subscribed to the Google Groups 
"Clojure" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to clojure+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: How to get an error message from Elastisch?

[lawrence . krubner]

This problem has become much stranger. Really, I thought I was developing 
some intuitions about how to write Clojure code, but everything about this 
seems counter-intuitive. 

When the app starts, it seems to be broken, doing one write per second to 
ElasticSearch (on AWS). Then I get an error, and the error seems to fix the 
app, which then starts to make about 15,000 writes a minute to 
ElasticSearch. The error is a retry attempt reported from either Apache 
Commons HTTP or from Elastish, which is the library I'm using to connect to 
ElasticSearch.

I am not sure that anyone can help me with this, but I'll share some 
details in case anyone has run into this before. 

I'm moving 1.2 million records from MySQL to ElasticSearch (it's actually 4 
million documents that aggregate down to 1.2 million). I'm using 
DurableQueue to move the records from one transformation to another, 
denormalizing the data into the format that the frontend needs. I use 3 
queues to move through our 3 transformations. I've also a function that 
runs in the background and every 30 seconds it prints out some facts about 
my app, to give me a rough idea of what is going on. I use the (stats) 
function provided by DurableQueue to see how the 3 queues are doing.  So 
after 4 or 5 minutes, I see this block of information: 

Resource usage:  Memory in use (percentage/used/max-heap): ("87%" "3131M" 
"3568M")

CPU usage (how-many-cpu's/load-average):  [4 3.88]

Free memory in jvm: [74424])

Stats about from-mysql-to-tables-queue:  {message {:num-slabs 1, 
:num-active-slabs 1, :enqueued 236358, :retried 0, :completed 236358, 
:in-progress -1}})
}

Stats about from-tables-to-topics-queue: {:num-slabs 1, :num-active-slabs 
1, :enqueued 45731, :retried  0, :completed 45729, :in-progress 1}}
 
Stats about the from-topics-to-persistence-queue:  {message {:num-slabs 1, 
:num-active-slabs 1, :enqueued 766, :retried 0, :completed 737, 
:in-progress 20}})

This final queue is moving at a glacial speed until I get these errors: 


Oct 04, 2017 10:27:35 PM org.apache.http.impl.client.DefaultHttpClient 
tryConnect
INFO: Retrying connect to 
{s}->https://search-samedayes01-ntsdht7rqwesdu.us-east-1.es.amazonaws.com:443

Oct 04, 2017 10:27:35 PM org.apache.http.impl.client.DefaultHttpClient 
tryConnect
INFO: I/O exception (java.net.SocketException) caught when connecting to 
{s}->https://search-samedayes01-ntsdht7rqwesdu.us-east-1.es.amazonaws.com:443: 
Broken pipe (Write failed)

Oct 04, 2017 10:27:35 PM org.apache.http.impl.client.DefaultHttpClient 
tryConnect
INFO: Retrying connect to 
{s}->https://search-samedayes01-ntsdht7rqwesdu.us-east-1.es.amazonaws.com:443

Strangely, these errors seem to fix my app. After I get these errors, the 
app starts writing to ElasticSearch at about 15,000 records per minute. 

A minute or two later I see:

Resource usage:  Memory in use (percentage/used/max-heap): ("62%" "2245M" 
"3568M")

CPU usage (how-many-cpu's/load-average):  [4 1.32]

Free memory in jvm: [611550400])

Stats about from-mysql-to-tables-queue:  {message {:num-slabs 1, 
:num-active-slabs 1, :enqueued 1156526, :retried 0, :completed 1156526, 
:in-progress 0}})

Stats about from-tables-to-topics-queue:  {message {:num-slabs 1, 
:num-active-slabs 1, :enqueued 59928, :retried 0, :completed 59928, 
:in-progress 0}})

Stats about the from-topics-to-persistence-queue:  {message {:num-slabs 1, 
:num-active-slabs 1, :enqueued 53222, :retried 0, :completed 53192, 
:in-progress 20}})


So memory use declines and the speed increases. 

It is possible that the java.net.SocketException is a red herring. It is 
possible that it shows up just when the app is done with most of the 
transformations, so of course memory use would go down and speed would go 
up. But I've run this several times and java.net.SocketException always 
shows up at the same time as the big increase in speed. 

I did some research about this error and ElasticSearch and I discovered 
this:

https://stackoverflow.com/questions/28908835/ssl-peer-shut-down-incorrectly-in-java

So I added this to the -main function that starts the app:

   (System/setProperty "https.protocols" "TLSv1.1")
   
This didn't seem to have any effect. 

Going out on a limb, it does seem that all the threads that write to 
ElasticSearch initially end up blocked, and then they timeout, and when 
they retry then things go well, so the app doesn't perform well until those 
threads writing to ElasticSearch have timedout and retried. But why the 
timeout should be necessary, and why the retry works so well (repeatedly, 
every time I do this) is beyond me. 

The code that spins up the worker threads that write to ElasticSearch looks 
like this: 

(defn advance
  [message db]
  {:pre [
 (= (type message) durable_queue.Task)
 ]}
  (persistence/push-item-to-persistence @message db)
  (durable/complete! message))


(defn worker
  [from-topics-to-persistence-queue db]
  (slingshot/try+
   (loop [message (durable

How to get an error message from Elastisch?

[lawrence . krubner]

This is probably a stupid question, but is there an obvious way to get an 
error message out of Elastisch? I had an app that was working with MongoDB 
and I was then told I had to use ElasticSearch instead (something about 
only using AWS for everything) so now I'm trying to get an error message, 
because my code doesn't seem to work. I read through here without seeing 
anything obvious: 

http://clojureelasticsearch.info/articles/getting_started.htm

I rewrote my MongoDB function, so it should work with Elastisch: 

(defn push-item-to-persistence
  [item db]
  (let [
denormalized-id (get-in item [:denormalized-id] :no-id)
item (assoc item :updated-at (temporal/current-time-as-datetime))
item (assoc item :permanent-holding-id-for-item-instances 
(java.util.UUID/randomUUID))
item (assoc item :instance-id-for-this-one-item 
(java.util.UUID/randomUUID))
item (assoc item :item-type :deduplication)
]
(if (= denormalized-id :no-id)
  (slingshot/throw+ {
 :type 
::no-denormalized-id-in-push-item-into-database
 :item item
 })
  (slingshot/try+
   (put conn index mapping-type id document)
   (esd/put db "facts-over-time" "deduplicaton" (str denormalized-id) 
item)
   (println " done with put in push-item-to-persistence ")
   (catch Object o
 (slingshot/throw+ {
:type ::push-item-to-persistence
:error o
:item item
:db db
}
   ))


It doesn't seem that any documents are getting into ElasticSearch. I was 
hoping to the (throw) would reveal to me some useful debugging information, 
but that doesn't seem to happen. 

The connection appears to be valid, as I can do this:

 (let [conn (persistence/multi-thread-start config)
   res  (esd/search conn "facts-over-time" 
"deduplication" :query {:match_all {}})
   n(esrsp/total-hits res)
   hits (esrsp/hits-from res)]
   (clojure.pprint/pprint res))

and I get: 

{:took 1,
 :timed_out false,
 :_shards {:total 5, :successful 5, :failed 0},
 :hits {:total 0, :max_score nil, :hits []}}

So the connection is there. But no records are. 


-- 
You received this message because you are subscribed to the Google
Groups "Clojure" group.
To post to this group, send email to clojure@googlegroups.com
Note that posts from new members are moderated - please be patient with your 
first post.
To unsubscribe from this group, send email to
clojure+unsubscr...@googlegroups.com
For more options, visit this group at
http://groups.google.com/group/clojure?hl=en
--- 
You received this message because you are subscribed to the Google Groups 
"Clojure" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to clojure+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: When does .getThreadCpuTime return nil?

[lawrence . krubner]

Thanks for the answer. I am surprised because I used this code for years 
without ever seeing that error. I'm trying to imagine what I did 
differently this one time. I've added error detection, we'll see if it 
happens again. 



On Friday, September 22, 2017 at 7:34:49 PM UTC-4, Neil Okamoto wrote:
>
>
> Maybe it's because you attempted method invocation via reflection but the 
> object is nil. So for example if (threads) returns a list containing nils, 
> then calling (.getId t) or (.getName t) would probably throw a null pointer 
> exception. I'm not sure it's possible for 
> (ManagementFactory/getThreadMXBean) to return nil, but if it does then 
> (.getThreadCpuTime mgr ...) would similarly fail.
>
> So best guess is, take a look at the implementation of "threads".
>
>
>
>
> On Friday, September 22, 2017 at 1:37:04 PM UTC-7, lawrence...@gmail.com 
> wrote:
>>
>> I've been using this code for years. I copy and pasted it from someone 
>> else on this group, someone much smarter than I am. I have never seen an 
>> error from this code. Now suddenly I get a null pointer exception at the 
>> last line of this code: 
>>
>> (ns denormalize_mysql_to_mongodb.monitoring
>>   (:import
>>java.lang.management.ManagementFactory)
>>   (:require [clojure.string :as st]))
>>
>>
>> (defn thread-top
>>   "Return a seq of threads sorted by their total userland CPU usage."
>>   []
>>   (let [mgr (ManagementFactory/getThreadMXBean)
>> cpu-times (map (fn [t]
>>  [(.getThreadCpuTime mgr (.getId t)) t])
>> (threads))]
>> (map
>>   (fn [[cpu t]] [cpu (.getName t) (.getId t) t])
>>   (reverse (sort-by first cpu-times)
>>
>> This error is thrown at startup. I don't know much about ThreadMXBean. 
>> Can anyone give me a clue why any of this would be nil? 
>>
>>
>>
>>
>>
>>

-- 
You received this message because you are subscribed to the Google
Groups "Clojure" group.
To post to this group, send email to clojure@googlegroups.com
Note that posts from new members are moderated - please be patient with your 
first post.
To unsubscribe from this group, send email to
clojure+unsubscr...@googlegroups.com
For more options, visit this group at
http://groups.google.com/group/clojure?hl=en
--- 
You received this message because you are subscribed to the Google Groups 
"Clojure" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to clojure+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

When does .getThreadCpuTime return nil?

[lawrence . krubner]

I've been using this code for years. I copy and pasted it from someone else 
on this group, someone much smarter than I am. I have never seen an error 
from this code. Now suddenly I get a null pointer exception at the last 
line of this code: 

(ns denormalize_mysql_to_mongodb.monitoring
  (:import
   java.lang.management.ManagementFactory)
  (:require [clojure.string :as st]))


(defn thread-top
  "Return a seq of threads sorted by their total userland CPU usage."
  []
  (let [mgr (ManagementFactory/getThreadMXBean)
cpu-times (map (fn [t]
 [(.getThreadCpuTime mgr (.getId t)) t])
(threads))]
(map
  (fn [[cpu t]] [cpu (.getName t) (.getId t) t])
  (reverse (sort-by first cpu-times)

This error is thrown at startup. I don't know much about ThreadMXBean. Can 
anyone give me a clue why any of this would be nil? 





-- 
You received this message because you are subscribed to the Google
Groups "Clojure" group.
To post to this group, send email to clojure@googlegroups.com
Note that posts from new members are moderated - please be patient with your 
first post.
To unsubscribe from this group, send email to
clojure+unsubscr...@googlegroups.com
For more options, visit this group at
http://groups.google.com/group/clojure?hl=en
--- 
You received this message because you are subscribed to the Google Groups 
"Clojure" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to clojure+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: does each class need its own file when I include Java files?

[lawrence . krubner]

Sorry, I totally misread it. Thank you so much. 


On Wednesday, September 20, 2017 at 3:55:40 AM UTC-4, Juraj Martinka wrote:
>
>
> > I believe I can copy-and-paste most of this code in the Java folder. Do 
> I need to put each class in its own file? 
>
> I'm not sure what you mean by "each class in its own file".
> In the example, there's one inner class. If you put this to java folder 
> everything should work.
> If you use leiningen, you can just specify `:java-source-paths 
> ["src/java"]` in your project.clj.
>
> I tested your example and it works pretty well - see here: 
> https://github.com/jumarko/clojure-repl-experiments/blob/master/src/clojure_repl_experiments/experiments.clj#L252
>
>
>
>
> On Tuesday, 19 September 2017 17:59:10 UTC+2, lawrence...@gmail.com wrote:
>>
>> I need to add something very much like this to my Clojure app:
>>
>>
>> https://svn.apache.org/repos/asf/poi/trunk/src/examples/src/org/apache/poi/xssf/eventusermodel/XLSX2CSV.java
>>
>> I currently have source paths set up as: 
>>
>> src/
>>  clojure/
>>  java/
>>
>>
>>
>>
>>
>>  
>>
>

-- 
You received this message because you are subscribed to the Google
Groups "Clojure" group.
To post to this group, send email to clojure@googlegroups.com
Note that posts from new members are moderated - please be patient with your 
first post.
To unsubscribe from this group, send email to
clojure+unsubscr...@googlegroups.com
For more options, visit this group at
http://groups.google.com/group/clojure?hl=en
--- 
You received this message because you are subscribed to the Google Groups 
"Clojure" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to clojure+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

does each class need its own file when I include Java files?

[lawrence . krubner]

I need to add something very much like this to my Clojure app:

https://svn.apache.org/repos/asf/poi/trunk/src/examples/src/org/apache/poi/xssf/eventusermodel/XLSX2CSV.java

I currently have source paths set up as: 

src/
 clojure/
 java/

I believe I can copy-and-paste most of this code in the Java folder. Do I 
need to put each class in its own file? 



 

-- 
You received this message because you are subscribed to the Google
Groups "Clojure" group.
To post to this group, send email to clojure@googlegroups.com
Note that posts from new members are moderated - please be patient with your 
first post.
To unsubscribe from this group, send email to
clojure+unsubscr...@googlegroups.com
For more options, visit this group at
http://groups.google.com/group/clojure?hl=en
--- 
You received this message because you are subscribed to the Google Groups 
"Clojure" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to clojure+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

how to be notified when a Future is realized?

[lawrence . krubner]

I stumbled across this old post by Tomasz Nurkiewicz:

http://www.nurkiewicz.com/2013/03/promises-and-futures-in-clojure.html

He writes: 

"And here is where the greatest disappointment arrives: neither future 
 nor promise 
 in Clojure 
supports listening for completion/failure asynchronously. The API is pretty 
much equivalent to very limited java.util.concurrent.Future 
. 
We can create future, cancel it 
, check 
whether it is realized? (resolved) 
 and block 
waiting for a value. Just like Future in Java, as a matter of fact the 
result of future function even implements java.util.concurrent.Future. 
As much as I love Clojure concurrency primitives like STM and agents, 
futures feel a bit underdeveloped. Lack of event-driven, asynchronous 
callbacks that are invoked whenever futures completes (notice that add-watch 
 doesn't work 
futures - and is still in alpha) greatly reduces the usefulness of a future 
object. "

That was written in 2013. I think since then the community has found other 
ways to achieve the same goals? I'm curious what patterns have become 
common? Would it be correct to say that for most of the use cases where one 
would otherwise want a notification of completion on a Future, people 
nowadays instead use something like core.async or a library such as 
Manifold? 








-- 
You received this message because you are subscribed to the Google
Groups "Clojure" group.
To post to this group, send email to clojure@googlegroups.com
Note that posts from new members are moderated - please be patient with your 
first post.
To unsubscribe from this group, send email to
clojure+unsubscr...@googlegroups.com
For more options, visit this group at
http://groups.google.com/group/clojure?hl=en
--- 
You received this message because you are subscribed to the Google Groups 
"Clojure" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to clojure+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: What is juxt really doing?

[lawrence . krubner]

Thank you for all the responses. The examples of using juxt to sort among 
results that are otherwise the same is a good example. 


On Sunday, July 16, 2017 at 3:18:07 AM UTC-4, Boris V. Schmid wrote:
>
> I don't use juxt much, but the example that I did pick up is where juxt is 
> used for sorting on one function first, and in the case of a tie, on the 
> second function. That is quite useful to me.
>
> > 
> (sort-by (juxt first second) (map vector (repeatedly 10 #(rand-int 3)) 
> (shuffle (range 10
> ([0 1] [0 4] [0 5] [0 6] [0 7] [0 8] [1 2] [1 3] [2 0] [2 9])
>
> On Sunday, July 16, 2017 at 5:52:44 AM UTC+2, lawrence...@gmail.com wrote:
>>
>>
>>
>> Does anyone use juxt in the real world, or is mostly for examples? 
>>
>>
>>
>  
>

-- 
You received this message because you are subscribed to the Google
Groups "Clojure" group.
To post to this group, send email to clojure@googlegroups.com
Note that posts from new members are moderated - please be patient with your 
first post.
To unsubscribe from this group, send email to
clojure+unsubscr...@googlegroups.com
For more options, visit this group at
http://groups.google.com/group/clojure?hl=en
--- 
You received this message because you are subscribed to the Google Groups 
"Clojure" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to clojure+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

What is juxt really doing?

[lawrence . krubner]

If I do this: 

((juxt :who :what :when) {:who 1 :when 2} {:who 4 :what 99}) 

I get: 

[1 {:who 4, :what 99} 2]

Why does a map come back instead of a number? 

Does anyone use juxt in the real world, or is mostly for examples? 


-- 
You received this message because you are subscribed to the Google
Groups "Clojure" group.
To post to this group, send email to clojure@googlegroups.com
Note that posts from new members are moderated - please be patient with your 
first post.
To unsubscribe from this group, send email to
clojure+unsubscr...@googlegroups.com
For more options, visit this group at
http://groups.google.com/group/clojure?hl=en
--- 
You received this message because you are subscribed to the Google Groups 
"Clojure" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to clojure+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: I can only get the first item of a lazyseq via a Manifold stream, and I can't get/find an Exception

[lawrence . krubner]

Sean, thanks for asking. The function is so basic that I didn't include it, 
but here it is:


(defn fetch [mysql-db]
  (slingshot/try+
   (jdbc/query mysql-db
   ["  SELECT p.id as profile_id,  p.name as profile_name, 
p.headquarters_addr1,  p.headquarters_city,  p.headquarters_state_code, 
headquarters_country_code, cw.url  FROM  company_profile p LEFT JOIN 
company_website cw ON p.id = cw.company_profile_id WHERE p.name is not null 
and p.name != ''  "])
   (catch Object o
 (errors/error o ""  " query to database had a problem "




On Wednesday, July 12, 2017 at 2:09:32 AM UTC-4, Sean Corfield wrote:
>
> What is the ‘fetch’ function here?
>
>  
>
> Sean Corfield -- (970) FOR-SEAN -- (904) 302-SEAN
> An Architect's View -- http://corfield.org/
>
> "If you're not annoying somebody, you're not really alive."
> -- Margaret Atwood
>
>  
>
> *From: *lawrence...@gmail.com 
> *Sent: *Tuesday, July 11, 2017 1:18 PM
> *To: *Clojure 
> *Subject: *Re: I can only get the first item of a lazyseq via a Manifold 
> stream, and I can't get/find an Exception
>
>  
>
> Justin, thanks. I'd like to ask a follow up question. To be clear, if I go 
> into (doseq) then this works fine:  
>
>  
>
> (defn start []
>
>   (let [
>
> config (get-config)
>
> mysql-db {
>
>   :dbtype "mysql"
>
>   :dbname (get-in  config [:database :config :connection 
> :database])
>
>   :user (get-in  config [:database :config :connection 
> :user] )
>
>   :password (get-in  config [:database :config :connection 
> :password] )
>
>   :host (get-in  config [:database :config :connection 
> :host])
>
>   }
>
> data (fetch mysql-db)
>
> ]
>
> (slingshot/try+
>
>  (doseq [row data]
>
>(queue/enqueue row))
>
>  (catch Object o
>
>(errors/error o ""  "error in query_database/start: ")
>
>  
>
>  
>
> but if I do this instead:
>
>  
>
> ;;;  (doseq [row data]
>
>(queue/enqueue data))
>
>
>
> and then enqueue does this: 
>
>  
>
>(->> (ms/->source data)
>
> (ms/onto executor)
>
> (ms/map api/query))
>
>  
>
>  
>
> The first row comes through fine, but then the second row is never 
> processed. 
>
>  
>
> So are you suggesting that simply passing "data" from one function to the 
> next is enough to lose the database context? But only after the first row 
> has been pulled? 
>
>  
>
>  
>
>  
>
>  
>
>  
>
>
> On Tuesday, July 11, 2017 at 1:44:15 PM UTC-4, Justin Smith wrote: 
>
> My first suspicion would be that by the time you access the second 
> element, you have exited the context of your database transaction, so 
> there's no data stream available to get it from. Lazyness doesn't tend to 
> mix well with stateful resources and contexts.
>
>  
>
> On Mon, Jul 10, 2017 at 9:45 PM  wrote:
>
> Okay, that was a deadend. After going through line by line, I could pretty 
> well rule out any kind of Exception or Error. The first row from the 
> database seems to go through all of the functions perfectly, without any 
> errors. But the second row never gets called.  
>
>  
>
> Which takes me back to where I started. Why can't I give a lazy-seq to a 
> Manifold Stream and simply have a function map over the stream? 
>
>
>
> On Monday, July 10, 2017 at 11:25:43 PM UTC-4, lawrence...@gmail.com 
> wrote: 
>
>  
>
> Once again, my lack of knowledge of Java trips me up. Manifold relies on 
> Dirigiste, which relies on Java's Executor Service. I see a bit here:
>
>  
>
> http://www.nurkiewicz.com/2014/11/executorservice-10-tips-and-tricks.html
>
>  
>
> Nurkiewicz writes:
>
> "I got bitten by that too many times: it won't print *anything*. No sign 
> of java.lang.ArithmeticException: / by zero, nothing. Thread pool just 
> swallows this exception, as if it never happened. If it was a good'ol 
> java.lang.Thread created from scratch, UncaughtExceptionHandler 
> 
>  could 
> work. "
>
>  
>
> I suspect I'm facing something like that. 
>
>  
>
>
>
> On Monday, July 10, 2017 at 8:28:03 PM UTC-4, lawrence...@gmail.com 
> wrote: 
>
> By the way, this code works fine if I go into a (doseq) a level above 
> enqueue, and then put each individual row onto the stream. Then the code 
> loops over all of the rows. But that seems to defeat the whole point of 
> using something like Manifold. I want to be able to put the whole lazy-seq 
> on the stream. That is supposed to work, yes?  
>
>
>
> On Monday, July 10, 2017 at 8:18:07 PM UTC-4, lawrence...@gmail.com 
> wrote: 
>
> I'm using Zach Tellman's excellent Manifold library, though I admit I 
> don't fully understand it.  
>
>  
>
> My code queries a MySQL database and then needs to do some processing on 
> each row retrieved. I copy-and-pasted some code from the documentation for 
> Manifold: 
>
>  
>
>  
>
> ;; 2017-07-10 -- we

Re: I can only get the first item of a lazyseq via a Manifold stream, and I can't get/find an Exception

[lawrence . krubner]

Justin, thanks. I'd like to ask a follow up question. To be clear, if I go 
into (doseq) then this works fine: 

(defn start []
  (let [
config (get-config)
mysql-db {
  :dbtype "mysql"
  :dbname (get-in  config [:database :config :connection 
:database])
  :user (get-in  config [:database :config :connection 
:user] )
  :password (get-in  config [:database :config :connection 
:password] )
  :host (get-in  config [:database :config :connection 
:host])
  }
data (fetch mysql-db)
]
(slingshot/try+
 (doseq [row data]
   (queue/enqueue row))
 (catch Object o
   (errors/error o ""  "error in query_database/start: ")


but if I do this instead:

;;;  (doseq [row data]
   (queue/enqueue data))
   
and then enqueue does this: 

   (->> (ms/->source data)
(ms/onto executor)
(ms/map api/query))


The first row comes through fine, but then the second row is never 
processed. 

So are you suggesting that simply passing "data" from one function to the 
next is enough to lose the database context? But only after the first row 
has been pulled? 






On Tuesday, July 11, 2017 at 1:44:15 PM UTC-4, Justin Smith wrote:
>
> My first suspicion would be that by the time you access the second 
> element, you have exited the context of your database transaction, so 
> there's no data stream available to get it from. Lazyness doesn't tend to 
> mix well with stateful resources and contexts.
>
> On Mon, Jul 10, 2017 at 9:45 PM > 
> wrote:
>
>> Okay, that was a deadend. After going through line by line, I could 
>> pretty well rule out any kind of Exception or Error. The first row from the 
>> database seems to go through all of the functions perfectly, without any 
>> errors. But the second row never gets called. 
>>
>> Which takes me back to where I started. Why can't I give a lazy-seq to a 
>> Manifold Stream and simply have a function map over the stream? 
>>
>>
>> On Monday, July 10, 2017 at 11:25:43 PM UTC-4, lawrence...@gmail.com 
>> wrote:
>>>
>>>
>>> Once again, my lack of knowledge of Java trips me up. Manifold relies on 
>>> Dirigiste, which relies on Java's Executor Service. I see a bit here:
>>>
>>> http://www.nurkiewicz.com/2014/11/executorservice-10-tips-and-tricks.html
>>>
>>> Nurkiewicz writes:
>>> "I got bitten by that too many times: it won't print *anything*. No 
>>> sign of java.lang.ArithmeticException: / by zero, nothing. Thread pool 
>>> just swallows this exception, as if it never happened. If it was a good'ol 
>>> java.lang.Thread created from scratch, UncaughtExceptionHandler 
>>> 
>>>  could 
>>> work. "
>>>
>>> I suspect I'm facing something like that. 
>>>
>>>
>>>
>>> On Monday, July 10, 2017 at 8:28:03 PM UTC-4, lawrence...@gmail.com 
>>> wrote:

 By the way, this code works fine if I go into a (doseq) a level above 
 enqueue, and then put each individual row onto the stream. Then the code 
 loops over all of the rows. But that seems to defeat the whole point of 
 using something like Manifold. I want to be able to put the whole lazy-seq 
 on the stream. That is supposed to work, yes? 


 On Monday, July 10, 2017 at 8:18:07 PM UTC-4, lawrence...@gmail.com 
 wrote:
>
> I'm using Zach Tellman's excellent Manifold library, though I admit I 
> don't fully understand it. 
>
> My code queries a MySQL database and then needs to do some processing 
> on each row retrieved. I copy-and-pasted some code from the documentation 
> for Manifold: 
>
>
> ;; 2017-07-10 -- we want a thread pool. I'm arbitrarily choosing 200 
> threads.
> ;; query_database/start will pull data from the database and dump it 
> onto a
> ;; stream below, at which point each row should be assigned to one of 
> the rows
> ;; on our thread pool. 
> (def executor (me/fixed-thread-executor 200))
>
>
> (defn enqueue
>   [sequence-from-database]
>   (slingshot/try+
>(println "the type of the object from the database: " (type 
> sequence-from-database))
>(->> (ms/->source sequence-from-database)
> (ms/onto executor)
> (ms/map api/query))
>(catch Object o
>  (println " message queue is not happy about the message we were 
> given")
>  (errors/error o "" " we tried to put something on the message 
> queue, but we got an error "
>
> The line where I print out the type assures that I'm dealing with a 
> LazySeq.
>
> The code prints out the type and the first row: 
>
>
> the type of the object from the database:  clojure.lang.LazySeq
>  
> in query_api/query  {:profile_id 2, :profile_name Mike Shaw Automotive 
> Group, :headquarters_addr1 90 

Re: I can only get the first item of a lazyseq via a Manifold stream, and I can't get/find an Exception

[lawrence . krubner]

Okay, that was a deadend. After going through line by line, I could pretty 
well rule out any kind of Exception or Error. The first row from the 
database seems to go through all of the functions perfectly, without any 
errors. But the second row never gets called. 

Which takes me back to where I started. Why can't I give a lazy-seq to a 
Manifold Stream and simply have a function map over the stream? 


On Monday, July 10, 2017 at 11:25:43 PM UTC-4, lawrence...@gmail.com wrote:
>
>
> Once again, my lack of knowledge of Java trips me up. Manifold relies on 
> Dirigiste, which relies on Java's Executor Service. I see a bit here:
>
> http://www.nurkiewicz.com/2014/11/executorservice-10-tips-and-tricks.html
>
> Nurkiewicz writes:
> "I got bitten by that too many times: it won't print *anything*. No sign 
> of java.lang.ArithmeticException: / by zero, nothing. Thread pool just 
> swallows this exception, as if it never happened. If it was a good'ol 
> java.lang.Thread created from scratch, UncaughtExceptionHandler 
> 
>  could 
> work. "
>
> I suspect I'm facing something like that. 
>
>
>
> On Monday, July 10, 2017 at 8:28:03 PM UTC-4, lawrence...@gmail.com wrote:
>>
>> By the way, this code works fine if I go into a (doseq) a level above 
>> enqueue, and then put each individual row onto the stream. Then the code 
>> loops over all of the rows. But that seems to defeat the whole point of 
>> using something like Manifold. I want to be able to put the whole lazy-seq 
>> on the stream. That is supposed to work, yes? 
>>
>>
>> On Monday, July 10, 2017 at 8:18:07 PM UTC-4, lawrence...@gmail.com 
>> wrote:
>>>
>>> I'm using Zach Tellman's excellent Manifold library, though I admit I 
>>> don't fully understand it. 
>>>
>>> My code queries a MySQL database and then needs to do some processing on 
>>> each row retrieved. I copy-and-pasted some code from the documentation for 
>>> Manifold: 
>>>
>>>
>>> ;; 2017-07-10 -- we want a thread pool. I'm arbitrarily choosing 200 
>>> threads.
>>> ;; query_database/start will pull data from the database and dump it 
>>> onto a
>>> ;; stream below, at which point each row should be assigned to one of 
>>> the rows
>>> ;; on our thread pool. 
>>> (def executor (me/fixed-thread-executor 200))
>>>
>>>
>>> (defn enqueue
>>>   [sequence-from-database]
>>>   (slingshot/try+
>>>(println "the type of the object from the database: " (type 
>>> sequence-from-database))
>>>(->> (ms/->source sequence-from-database)
>>> (ms/onto executor)
>>> (ms/map api/query))
>>>(catch Object o
>>>  (println " message queue is not happy about the message we were 
>>> given")
>>>  (errors/error o "" " we tried to put something on the message 
>>> queue, but we got an error "
>>>
>>> The line where I print out the type assures that I'm dealing with a 
>>> LazySeq.
>>>
>>> The code prints out the type and the first row: 
>>>
>>>
>>> the type of the object from the database:  clojure.lang.LazySeq
>>>  
>>> in query_api/query  {:profile_id 2, :profile_name Mike Shaw Automotive 
>>> Group, :headquarters_addr1 90 Madison St., :headquarters_city Denver, 
>>> :headquarters_state_code CO, :headquarters_country_code US, :url 
>>> mikeshawauto.com}
>>>
>>> and then it stops. I assume there must be an Exception or Error 
>>> happening, but I can't find it. I've added as many general Catch clauses as 
>>> I could: 
>>>
>>>
>>> (defn query
>>>   [row & args]
>>>
>>>   (println " in query_api/query " row)
>>>   
>>>   (let [config (if args
>>>  (first args))
>>> ]
>>> ;; 2017-03-30 -- the API is overwhelmed and all I get is Socket 
>>> Timeout errors
>>> (Thread/sleep 300)
>>> 
>>> (slingshot/try+
>>>  (call-api row)
>>>  (catch Object o
>>>(println " error : " o)
>>>
>>>;;(errors/error o row " we tried to call-api, but we got this 
>>> error ")
>>>
>>>
>>>)
>>>
>>>  (catch Error e
>>>(println " there Error: " e))
>>>  )))
>>>
>>> So if I do: 
>>>
>>> java  -jar scan-database-find-similar-items-standalone.jar 
>>>
>>>
>>> The code runs till it prints out the first row from the database, and 
>>> then it stops. Nothing else happens. There are no error messages. 
>>>
>>> What did I miss? 
>>>
>>>
>>>
>>>
>>>
>>>
>>>
>>>

-- 
You received this message because you are subscribed to the Google
Groups "Clojure" group.
To post to this group, send email to clojure@googlegroups.com
Note that posts from new members are moderated - please be patient with your 
first post.
To unsubscribe from this group, send email to
clojure+unsubscr...@googlegroups.com
For more options, visit this group at
http://groups.google.com/group/clojure?hl=en
--- 
You received this message because you are subscribed to the Google Groups 
"Clojure" group.
To unsubscribe from this group and stop receiving emails from it, send an ema

Re: I can only get the first item of a lazyseq via a Manifold stream, and I can't get/find an Exception

[lawrence . krubner]

Once again, my lack of knowledge of Java trips me up. Manifold relies on 
Dirigiste, which relies on Java's Executor Service. I see a bit here:

http://www.nurkiewicz.com/2014/11/executorservice-10-tips-and-tricks.html

Nurkiewicz writes:
"I got bitten by that too many times: it won't print *anything*. No sign of 
java.lang.ArithmeticException: 
/ by zero, nothing. Thread pool just swallows this exception, as if it 
never happened. If it was a good'ol java.lang.Thread created from scratch, 
UncaughtExceptionHandler 

 could 
work. "

I suspect I'm facing something like that. 



On Monday, July 10, 2017 at 8:28:03 PM UTC-4, lawrence...@gmail.com wrote:
>
> By the way, this code works fine if I go into a (doseq) a level above 
> enqueue, and then put each individual row onto the stream. Then the code 
> loops over all of the rows. But that seems to defeat the whole point of 
> using something like Manifold. I want to be able to put the whole lazy-seq 
> on the stream. That is supposed to work, yes? 
>
>
> On Monday, July 10, 2017 at 8:18:07 PM UTC-4, lawrence...@gmail.com wrote:
>>
>> I'm using Zach Tellman's excellent Manifold library, though I admit I 
>> don't fully understand it. 
>>
>> My code queries a MySQL database and then needs to do some processing on 
>> each row retrieved. I copy-and-pasted some code from the documentation for 
>> Manifold: 
>>
>>
>> ;; 2017-07-10 -- we want a thread pool. I'm arbitrarily choosing 200 
>> threads.
>> ;; query_database/start will pull data from the database and dump it onto 
>> a
>> ;; stream below, at which point each row should be assigned to one of the 
>> rows
>> ;; on our thread pool. 
>> (def executor (me/fixed-thread-executor 200))
>>
>>
>> (defn enqueue
>>   [sequence-from-database]
>>   (slingshot/try+
>>(println "the type of the object from the database: " (type 
>> sequence-from-database))
>>(->> (ms/->source sequence-from-database)
>> (ms/onto executor)
>> (ms/map api/query))
>>(catch Object o
>>  (println " message queue is not happy about the message we were 
>> given")
>>  (errors/error o "" " we tried to put something on the message queue, 
>> but we got an error "
>>
>> The line where I print out the type assures that I'm dealing with a 
>> LazySeq.
>>
>> The code prints out the type and the first row: 
>>
>>
>> the type of the object from the database:  clojure.lang.LazySeq
>>  
>> in query_api/query  {:profile_id 2, :profile_name Mike Shaw Automotive 
>> Group, :headquarters_addr1 90 Madison St., :headquarters_city Denver, 
>> :headquarters_state_code CO, :headquarters_country_code US, :url 
>> mikeshawauto.com}
>>
>> and then it stops. I assume there must be an Exception or Error 
>> happening, but I can't find it. I've added as many general Catch clauses as 
>> I could: 
>>
>>
>> (defn query
>>   [row & args]
>>
>>   (println " in query_api/query " row)
>>   
>>   (let [config (if args
>>  (first args))
>> ]
>> ;; 2017-03-30 -- the API is overwhelmed and all I get is Socket 
>> Timeout errors
>> (Thread/sleep 300)
>> 
>> (slingshot/try+
>>  (call-api row)
>>  (catch Object o
>>(println " error : " o)
>>
>>;;(errors/error o row " we tried to call-api, but we got this 
>> error ")
>>
>>
>>)
>>
>>  (catch Error e
>>(println " there Error: " e))
>>  )))
>>
>> So if I do: 
>>
>> java  -jar scan-database-find-similar-items-standalone.jar 
>>
>>
>> The code runs till it prints out the first row from the database, and 
>> then it stops. Nothing else happens. There are no error messages. 
>>
>> What did I miss? 
>>
>>
>>
>>
>>
>>
>>
>>

-- 
You received this message because you are subscribed to the Google
Groups "Clojure" group.
To post to this group, send email to clojure@googlegroups.com
Note that posts from new members are moderated - please be patient with your 
first post.
To unsubscribe from this group, send email to
clojure+unsubscr...@googlegroups.com
For more options, visit this group at
http://groups.google.com/group/clojure?hl=en
--- 
You received this message because you are subscribed to the Google Groups 
"Clojure" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to clojure+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: I can only get the first item of a lazyseq via a Manifold stream, and I can't get/find an Exception

[lawrence . krubner]

By the way, this code works fine if I go into a (doseq) a level above 
enqueue, and then put each individual row onto the stream. Then the code 
loops over all of the rows. But that seems to defeat the whole point of 
using something like Manifold. I want to be able to put the whole lazy-seq 
on the stream. That is supposed to work, yes? 


On Monday, July 10, 2017 at 8:18:07 PM UTC-4, lawrence...@gmail.com wrote:
>
> I'm using Zach Tellman's excellent Manifold library, though I admit I 
> don't fully understand it. 
>
> My code queries a MySQL database and then needs to do some processing on 
> each row retrieved. I copy-and-pasted some code from the documentation for 
> Manifold: 
>
>
> ;; 2017-07-10 -- we want a thread pool. I'm arbitrarily choosing 200 
> threads.
> ;; query_database/start will pull data from the database and dump it onto a
> ;; stream below, at which point each row should be assigned to one of the 
> rows
> ;; on our thread pool. 
> (def executor (me/fixed-thread-executor 200))
>
>
> (defn enqueue
>   [sequence-from-database]
>   (slingshot/try+
>(println "the type of the object from the database: " (type 
> sequence-from-database))
>(->> (ms/->source sequence-from-database)
> (ms/onto executor)
> (ms/map api/query))
>(catch Object o
>  (println " message queue is not happy about the message we were 
> given")
>  (errors/error o "" " we tried to put something on the message queue, 
> but we got an error "
>
> The line where I print out the type assures that I'm dealing with a 
> LazySeq.
>
> The code prints out the type and the first row: 
>
>
> the type of the object from the database:  clojure.lang.LazySeq
>  
> in query_api/query  {:profile_id 2, :profile_name Mike Shaw Automotive 
> Group, :headquarters_addr1 90 Madison St., :headquarters_city Denver, 
> :headquarters_state_code CO, :headquarters_country_code US, :url 
> mikeshawauto.com}
>
> and then it stops. I assume there must be an Exception or Error happening, 
> but I can't find it. I've added as many general Catch clauses as I could: 
>
>
> (defn query
>   [row & args]
>
>   (println " in query_api/query " row)
>   
>   (let [config (if args
>  (first args))
> ]
> ;; 2017-03-30 -- the API is overwhelmed and all I get is Socket 
> Timeout errors
> (Thread/sleep 300)
> 
> (slingshot/try+
>  (call-api row)
>  (catch Object o
>(println " error : " o)
>
>;;(errors/error o row " we tried to call-api, but we got this error 
> ")
>
>
>)
>
>  (catch Error e
>(println " there Error: " e))
>  )))
>
> So if I do: 
>
> java  -jar scan-database-find-similar-items-standalone.jar 
>
>
> The code runs till it prints out the first row from the database, and then 
> it stops. Nothing else happens. There are no error messages. 
>
> What did I miss? 
>
>
>
>
>
>
>
>

-- 
You received this message because you are subscribed to the Google
Groups "Clojure" group.
To post to this group, send email to clojure@googlegroups.com
Note that posts from new members are moderated - please be patient with your 
first post.
To unsubscribe from this group, send email to
clojure+unsubscr...@googlegroups.com
For more options, visit this group at
http://groups.google.com/group/clojure?hl=en
--- 
You received this message because you are subscribed to the Google Groups 
"Clojure" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to clojure+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

I can only get the first item of a lazyseq via a Manifold stream, and I can't get/find an Exception

[lawrence . krubner]

I'm using Zach Tellman's excellent Manifold library, though I admit I don't 
fully understand it. 

My code queries a MySQL database and then needs to do some processing on 
each row retrieved. I copy-and-pasted some code from the documentation for 
Manifold: 


;; 2017-07-10 -- we want a thread pool. I'm arbitrarily choosing 200 
threads.
;; query_database/start will pull data from the database and dump it onto a
;; stream below, at which point each row should be assigned to one of the 
rows
;; on our thread pool. 
(def executor (me/fixed-thread-executor 200))


(defn enqueue
  [sequence-from-database]
  (slingshot/try+
   (println "the type of the object from the database: " (type 
sequence-from-database))
   (->> (ms/->source sequence-from-database)
(ms/onto executor)
(ms/map api/query))
   (catch Object o
 (println " message queue is not happy about the message we were given")
 (errors/error o "" " we tried to put something on the message queue, 
but we got an error "

The line where I print out the type assures that I'm dealing with a LazySeq.

The code prints out the type and the first row: 


the type of the object from the database:  clojure.lang.LazySeq
 
in query_api/query  {:profile_id 2, :profile_name Mike Shaw Automotive 
Group, :headquarters_addr1 90 Madison St., :headquarters_city Denver, 
:headquarters_state_code CO, :headquarters_country_code US, :url 
mikeshawauto.com}

and then it stops. I assume there must be an Exception or Error happening, 
but I can't find it. I've added as many general Catch clauses as I could: 


(defn query
  [row & args]

  (println " in query_api/query " row)
  
  (let [config (if args
 (first args))
]
;; 2017-03-30 -- the API is overwhelmed and all I get is Socket Timeout 
errors
(Thread/sleep 300)

(slingshot/try+
 (call-api row)
 (catch Object o
   (println " error : " o)
   
   ;;(errors/error o row " we tried to call-api, but we got this error 
")


   )

 (catch Error e
   (println " there Error: " e))
 )))

So if I do: 

java  -jar scan-database-find-similar-items-standalone.jar 


The code runs till it prints out the first row from the database, and then 
it stops. Nothing else happens. There are no error messages. 

What did I miss? 







-- 
You received this message because you are subscribed to the Google
Groups "Clojure" group.
To post to this group, send email to clojure@googlegroups.com
Note that posts from new members are moderated - please be patient with your 
first post.
To unsubscribe from this group, send email to
clojure+unsubscr...@googlegroups.com
For more options, visit this group at
http://groups.google.com/group/clojure?hl=en
--- 
You received this message because you are subscribed to the Google Groups 
"Clojure" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to clojure+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: java.lang.ClassNotFoundException: clojure.lang.Var

[lawrence . krubner]

Sorry, that was dumb. Obviously I meant to call with classpath. It works 
fine with standalone. 



On Monday, July 10, 2017 at 1:27:53 AM UTC-4, lawrence...@gmail.com wrote:
>
> I was trying to knock out a quick project today, and most of it was 
> copy-and-paste of another project that I have. This isn't really a big 
> project, despite the long list of dependencies (which I could trim 
> somewhat). 
>
> Anyway, if I run "lein clean" and then "lein uberjar" it seems to compile 
> just fine. 
>
> But then I do this: 
>
>  java -jar target/scan-database-find-similar-items.jar 
>
> and get this: 
>
> Exception in thread "main" java.lang.NoClassDefFoundError: clojure/lang/Var
> at scan_database_find_similar_items.core.(Unknown Source)
> Caused by: java.lang.ClassNotFoundException: clojure.lang.Var
> at java.net.URLClassLoader$1.run(URLClassLoader.java:372)
> at java.net.URLClassLoader$1.run(URLClassLoader.java:361)
> at java.security.AccessController.doPrivileged(Native Method)
> at java.net.URLClassLoader.findClass(URLClassLoader.java:360)
> at java.lang.ClassLoader.loadClass(ClassLoader.java:424)
> at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:308)
> at java.lang.ClassLoader.loadClass(ClassLoader.java:357)
> ... 1 more
>
> What would cause this?
>
> Below is my project.clj file
>
>
> (defproject scan-database-find-similar-items "1.0"
>   :description "scan-database-find-similar-items compares items in the 
> database to other items in the database, in the hopes of finding fake 
> profiles that are duplicates. "
>   :url "https://github.com/jazz/scan-database-find-similar-items";
>   :license {:name "Copyright Sameday.com 2017"
> :url "http://www.sameday.com/"}
>   :dependencies [
>  [org.clojure/clojure "1.8.0"]
>  [org.clojure/test.check "0.9.0"]
>  [org.clojure/java.jdbc "0.7.0-alpha1"]
>  [org.clojure/data.json "0.2.5"]
>  [org.clojure/tools.namespace "0.2.4"]
>  [org.clojure/core.incubator "0.1.3"]
>  [org.clojure/core.match "0.3.0-alpha4"]
>  
>  [clj-stacktrace "0.2.7"]
>  [clj-time "0.6.0"]
>  
>  [joda-time/joda-time "2.9.4"]
>
>  [com.taoensso/timbre "4.3.1"]
>  [defun "0.3.0-RC1"]
>  [slingshot "0.12.2"]
>  [manifold "0.1.2"]
>  [me.raynes/fs "1.4.4"]
>  [overtone/at-at "1.2.0"]
>  [mysql/mysql-connector-java "6.0.5"]
>  [com.novemberain/monger "3.1.0"]
>  [cheshire "5.3.1"]
>  
>  ;; TODO: remove one of these. both are in use
>  [clj-http "2.3.0"]
>  [http-kit "2.2.0"]
>  ]
>
>   :plugins [
> [lein-localrepo "0.5.3"] ;; install java artifacts locally, 
> even using fake artifactId
> [lein-cprint "1.0.0"] ;; the same as lein-pprint but with 
> colors
> [lein-exec "0.3.5"] ;; execute text as Clojure code
> [lein-nevam "0.1.2"] ;; convert a Maven pom to a project.clj
> [lein-vanity "0.2.0"] ;; lines of code for vanity's sake
> [lein-expectations "0.0.8"] ;; run expectations tests
> [lein-collisions "0.1.4"] ;; find classpath collisions
> ]
>
>   :aot :all  
>   :source-paths  ["src/clojure"]
>   :java-source-paths ["src/java"]
>   :resource-paths ["src/main/resource"] ; Non-code files included in 
> classpath/jar.
>
>   ;; Name of the jar file produced. Will be placed inside :target-path.
>   ;; Including %s will splice the project version into the filename.
>   :jar-name "scan-database-find-similar-items.jar"
>   ;; As above, but for uberjar.
>   :uberjar-name "scan-database-find-similar-items-standalone.jar"
>   
>   ;; What to do in the case of version issues. Defaults to :ranges, which
>   ;; warns when version ranges are present anywhere in the dependency tree,
>   ;; but can be set to true to warn for both ranges and overrides, or 
> :abort
>   ;; to exit in the case of ranges or overrides.
>   ;;  :pedantic? :abort
>   
>
>   :disable-implicit-clean true
>   :warn-on-reflection true
>   :main scan-database-find-similar-items.core
>   :jvm-opts ["-Xms100m" "-Xmx1000m" "-XX:-UseCompressedOops"])
>
>
>
>

-- 
You received this message because you are subscribed to the Google
Groups "Clojure" group.
To post to this group, send email to clojure@googlegroups.com
Note that posts from new members are moderated - please be patient with your 
first post.
To unsubscribe from this group, send email to
clojure+unsubscr...@googlegroups.com
For more options, visit this group at
http://groups.google.com/group/clojure?hl=en
--- 
You received this message because you are subscribed to the Google Groups 
"Clojure" group.
To unsubscribe from this group and stop receiving emai

java.lang.ClassNotFoundException: clojure.lang.Var

[lawrence . krubner]

I was trying to knock out a quick project today, and most of it was 
copy-and-paste of another project that I have. This isn't really a big 
project, despite the long list of dependencies (which I could trim 
somewhat). 

Anyway, if I run "lein clean" and then "lein uberjar" it seems to compile 
just fine. 

But then I do this: 

 java -jar target/scan-database-find-similar-items.jar 

and get this: 

Exception in thread "main" java.lang.NoClassDefFoundError: clojure/lang/Var
at scan_database_find_similar_items.core.(Unknown Source)
Caused by: java.lang.ClassNotFoundException: clojure.lang.Var
at java.net.URLClassLoader$1.run(URLClassLoader.java:372)
at java.net.URLClassLoader$1.run(URLClassLoader.java:361)
at java.security.AccessController.doPrivileged(Native Method)
at java.net.URLClassLoader.findClass(URLClassLoader.java:360)
at java.lang.ClassLoader.loadClass(ClassLoader.java:424)
at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:308)
at java.lang.ClassLoader.loadClass(ClassLoader.java:357)
... 1 more

What would cause this?

Below is my project.clj file


(defproject scan-database-find-similar-items "1.0"
  :description "scan-database-find-similar-items compares items in the 
database to other items in the database, in the hopes of finding fake 
profiles that are duplicates. "
  :url "https://github.com/jazz/scan-database-find-similar-items";
  :license {:name "Copyright Sameday.com 2017"
:url "http://www.sameday.com/"}
  :dependencies [
 [org.clojure/clojure "1.8.0"]
 [org.clojure/test.check "0.9.0"]
 [org.clojure/java.jdbc "0.7.0-alpha1"]
 [org.clojure/data.json "0.2.5"]
 [org.clojure/tools.namespace "0.2.4"]
 [org.clojure/core.incubator "0.1.3"]
 [org.clojure/core.match "0.3.0-alpha4"]
 
 [clj-stacktrace "0.2.7"]
 [clj-time "0.6.0"]
 
 [joda-time/joda-time "2.9.4"]

 [com.taoensso/timbre "4.3.1"]
 [defun "0.3.0-RC1"]
 [slingshot "0.12.2"]
 [manifold "0.1.2"]
 [me.raynes/fs "1.4.4"]
 [overtone/at-at "1.2.0"]
 [mysql/mysql-connector-java "6.0.5"]
 [com.novemberain/monger "3.1.0"]
 [cheshire "5.3.1"]
 
 ;; TODO: remove one of these. both are in use
 [clj-http "2.3.0"]
 [http-kit "2.2.0"]
 ]

  :plugins [
[lein-localrepo "0.5.3"] ;; install java artifacts locally, 
even using fake artifactId
[lein-cprint "1.0.0"] ;; the same as lein-pprint but with colors
[lein-exec "0.3.5"] ;; execute text as Clojure code
[lein-nevam "0.1.2"] ;; convert a Maven pom to a project.clj
[lein-vanity "0.2.0"] ;; lines of code for vanity's sake
[lein-expectations "0.0.8"] ;; run expectations tests
[lein-collisions "0.1.4"] ;; find classpath collisions
]

  :aot :all  
  :source-paths  ["src/clojure"]
  :java-source-paths ["src/java"]
  :resource-paths ["src/main/resource"] ; Non-code files included in 
classpath/jar.

  ;; Name of the jar file produced. Will be placed inside :target-path.
  ;; Including %s will splice the project version into the filename.
  :jar-name "scan-database-find-similar-items.jar"
  ;; As above, but for uberjar.
  :uberjar-name "scan-database-find-similar-items-standalone.jar"
  
  ;; What to do in the case of version issues. Defaults to :ranges, which
  ;; warns when version ranges are present anywhere in the dependency tree,
  ;; but can be set to true to warn for both ranges and overrides, or :abort
  ;; to exit in the case of ranges or overrides.
  ;;  :pedantic? :abort
  

  :disable-implicit-clean true
  :warn-on-reflection true
  :main scan-database-find-similar-items.core
  :jvm-opts ["-Xms100m" "-Xmx1000m" "-XX:-UseCompressedOops"])



-- 
You received this message because you are subscribed to the Google
Groups "Clojure" group.
To post to this group, send email to clojure@googlegroups.com
Note that posts from new members are moderated - please be patient with your 
first post.
To unsubscribe from this group, send email to
clojure+unsubscr...@googlegroups.com
For more options, visit this group at
http://groups.google.com/group/clojure?hl=en
--- 
You received this message because you are subscribed to the Google Groups 
"Clojure" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to clojure+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: error in nrepl

[lawrence . krubner]

Thank you, Didier, but I'll wait till ProtoRepl is more stable. Also I'm 
not sure that I'm ready to step away from Emacs. 

I did try Spacemacs but I felt it was too much influenced by Vim. So now 
I'm using Prelude. 





On Thursday, June 29, 2017 at 1:46:11 PM UTC-4, Didier wrote:
>
> If you're no fan on emacs or vim, ProtoRepl is great. I also recommend 
> cursive, but if you're no fan of intelliJ autosave, counterclockwise 
> eclipse os surprisingly great. 

-- 
You received this message because you are subscribed to the Google
Groups "Clojure" group.
To post to this group, send email to clojure@googlegroups.com
Note that posts from new members are moderated - please be patient with your 
first post.
To unsubscribe from this group, send email to
clojure+unsubscr...@googlegroups.com
For more options, visit this group at
http://groups.google.com/group/clojure?hl=en
--- 
You received this message because you are subscribed to the Google Groups 
"Clojure" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to clojure+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: Clojure versions used by java.jdbc users

[lawrence . krubner]

I have some old projects that ran Clojure 1.5 and jbcd. I wrote them before 
the era of Cider. Cider only supports Clojure => 1.7. I do have to now use 
those old projects, but I was planning on upgrading them anyway, so I could 
work on them in Cider. 




On Monday, July 3, 2017 at 7:12:19 PM UTC-4, Sean Corfield wrote:
>
> I ran a short survey for java.jdbc users to gauge feeling about dropping 
> support for Clojure versions prior to 1.7 (so I could add reducible queries 
> without worrying about CollReduce vs IReduceInit vs IReduce).
>
>  
>
> So far, 68 people have taken the survey and the results are overwhelmingly 
> in favor of only supporting Clojure 1.7+ -- so I thought I’d share the 
> results with a larger audience, just for information:
>
>  
>
> https://www.surveymonkey.com/results/SM-CJY2YMHP/
>
>  
>
> Of 68 respondents, only one is still on Clojure 1.7, none are on earlier 
> versions. I was expecting a few more 1.7 responses – and I was not 
> expecting 35% already on Clojure 1.9 alpha builds!
>
>  
>
> If you use java.jdbc and haven’t already taken the survey and want your 
> voice heard:
>
>  
>
> https://www.surveymonkey.com/r/MR2HRFD
>
>  
>
> Sean Corfield -- (904) 302-SEAN -- (970) FOR-SEAN
>
> An Architect's View -- http://corfield.org/
>
>  
>
> "Perfection is the enemy of the good."
>
> -- Gustave Flaubert, French realist novelist (1821-1880)
>
>  
>
>  
>

-- 
You received this message because you are subscribed to the Google
Groups "Clojure" group.
To post to this group, send email to clojure@googlegroups.com
Note that posts from new members are moderated - please be patient with your 
first post.
To unsubscribe from this group, send email to
clojure+unsubscr...@googlegroups.com
For more options, visit this group at
http://groups.google.com/group/clojure?hl=en
--- 
You received this message because you are subscribed to the Google Groups 
"Clojure" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to clojure+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: error in nrepl

[lawrence . krubner]

Thank you for everyone's reply. Both Spacemacs and ProtoRepl sound very 
interesting. I am torn and am unsure which path to take. I suppose I'll 
give Spacemacs a try and see if that works out.

I am very pleased to see progress being made regarding at least the DE of 
IDE for Clojure. That's always been a pain point. 



On Monday, June 26, 2017 at 12:44:45 PM UTC-4, Bozhidar Batsov wrote:
>
> Newer versions of CIDER are much easier to setup than the older ones (e.g. 
> they auto-inject their dependencies, so you don't have to fiddle with 
> profiles.clj). You should try the latest stable or dev release. That said 
> you can also check Monroe (https://github.com/sanel/monroe) which is a 
> fork of a very old version of CIDER (from around the time it was named 
> nrepl.el) or inf-clojure (https://github.com/clojure-emacs/inf-clojure) - 
> a completely 0-setup Clojure(Script) REPL with support for connecting to a 
> REPL socket server. 
>
> On 24 June 2017 at 00:36, > wrote:
>
>>
>> Yes, sadly, I've never gotten Cider to work with Emacs. I keep thinking 
>> someday I'll take a weekend and work through all the errors and get it 
>> working, but I never seem to find the time. So I keep working with an old 
>> version of nrepl. But I take it, from your answer, you think this error 
>> would vanish if I upgraded to Cider? 
>>
>>
>>
>>
>> On Friday, June 23, 2017 at 5:15:09 PM UTC-4, James Reeves wrote:
>>>
>>> nrepl-jack-in? Do you mean cider-jack-in? AFAIK nrepl-jack-in is from a 
>>> very old version of Cider.
>>>
>>> On 23 June 2017 at 21:29,  wrote:
>>>
 I'm using Emacs on my Mac. I ran "nrepl-jack-in" to load up the repl. 
 I'm iterating over a dataset from mysql. My code is very simple, I'm just 
 trying to count the words: 

 (reduce 

 (fn [map-of-word-count next-name] 
 (let [
 words (clojure.string/split next-name #"\s") 
 map-of-names-words-with-count (frequencies words)
 ] 
 (println map-of-names-words-with-count)
 (merge-with + map-of-word-count map-of-names-words-with-count)
 )
 ) 
 {} 
 names)

 I keep getting this message:


 error in process filter: nrepl-bdecode-buffer: Cannot decode object: 1
 error in process filter: Cannot decode object: 1
 Error running timer `jit-lock-stealth-fontify': (error "Variable 
 binding depth exceeds max-specpdl-size")
 timer-relative-time: Variable binding depth exceeds max-specpdl-size


 Does anyone know what this means? 

 -- 
 You received this message because you are subscribed to the Google
 Groups "Clojure" group.
 To post to this group, send email to clo...@googlegroups.com
 Note that posts from new members are moderated - please be patient with 
 your first post.
 To unsubscribe from this group, send email to
 clojure+u...@googlegroups.com
 For more options, visit this group at
 http://groups.google.com/group/clojure?hl=en
 --- 
 You received this message because you are subscribed to the Google 
 Groups "Clojure" group.
 To unsubscribe from this group and stop receiving emails from it, send 
 an email to clojure+u...@googlegroups.com.
 For more options, visit https://groups.google.com/d/optout.

>>>
>>>
>>>
>>> -- 
>>> James Reeves
>>> booleanknot.com
>>>
>> -- 
>> You received this message because you are subscribed to the Google
>> Groups "Clojure" group.
>> To post to this group, send email to clo...@googlegroups.com 
>> 
>> Note that posts from new members are moderated - please be patient with 
>> your first post.
>> To unsubscribe from this group, send email to
>> clojure+u...@googlegroups.com 
>> For more options, visit this group at
>> http://groups.google.com/group/clojure?hl=en
>> --- 
>> You received this message because you are subscribed to the Google Groups 
>> "Clojure" group.
>> To unsubscribe from this group and stop receiving emails from it, send an 
>> email to clojure+u...@googlegroups.com .
>> For more options, visit https://groups.google.com/d/optout.
>>
>
>

-- 
You received this message because you are subscribed to the Google
Groups "Clojure" group.
To post to this group, send email to clojure@googlegroups.com
Note that posts from new members are moderated - please be patient with your 
first post.
To unsubscribe from this group, send email to
clojure+unsubscr...@googlegroups.com
For more options, visit this group at
http://groups.google.com/group/clojure?hl=en
--- 
You received this message because you are subscribed to the Google Groups 
"Clojure" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to clojure+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: error in nrepl

[lawrence . krubner]

Thank you. Maybe I can find some time to upgrade my whole Emacs setup next 
weekend. It is a bit out of date. 

I'm curious if folks think it is easier to work with Emacs on a Linux 
machine, or on a Mac? 



On Friday, June 23, 2017 at 5:39:42 PM UTC-4, Kevin Baldor wrote:
>
> Have you tried following the instructions at 
> http://www.braveclojure.com/basic-emacs/ ?
>
> It's a bit heavy-handed (replacing your entire .emacs directory), but it 
> might give you a starting point to figure out how to integrate it into your 
> emacs setup.
>
> On Fri, Jun 23, 2017 at 4:36 PM, > 
> wrote:
>
>>
>> Yes, sadly, I've never gotten Cider to work with Emacs. I keep thinking 
>> someday I'll take a weekend and work through all the errors and get it 
>> working, but I never seem to find the time. So I keep working with an old 
>> version of nrepl. But I take it, from your answer, you think this error 
>> would vanish if I upgraded to Cider? 
>>
>>
>>
>>
>> On Friday, June 23, 2017 at 5:15:09 PM UTC-4, James Reeves wrote:
>>>
>>> nrepl-jack-in? Do you mean cider-jack-in? AFAIK nrepl-jack-in is from a 
>>> very old version of Cider.
>>>
>>> On 23 June 2017 at 21:29,  wrote:
>>>
 I'm using Emacs on my Mac. I ran "nrepl-jack-in" to load up the repl. 
 I'm iterating over a dataset from mysql. My code is very simple, I'm just 
 trying to count the words: 

 (reduce 

 (fn [map-of-word-count next-name] 
 (let [
 words (clojure.string/split next-name #"\s") 
 map-of-names-words-with-count (frequencies words)
 ] 
 (println map-of-names-words-with-count)
 (merge-with + map-of-word-count map-of-names-words-with-count)
 )
 ) 
 {} 
 names)

 I keep getting this message:


 error in process filter: nrepl-bdecode-buffer: Cannot decode object: 1
 error in process filter: Cannot decode object: 1
 Error running timer `jit-lock-stealth-fontify': (error "Variable 
 binding depth exceeds max-specpdl-size")
 timer-relative-time: Variable binding depth exceeds max-specpdl-size


 Does anyone know what this means? 

 -- 
 You received this message because you are subscribed to the Google
 Groups "Clojure" group.
 To post to this group, send email to clo...@googlegroups.com
 Note that posts from new members are moderated - please be patient with 
 your first post.
 To unsubscribe from this group, send email to
 clojure+u...@googlegroups.com
 For more options, visit this group at
 http://groups.google.com/group/clojure?hl=en
 --- 
 You received this message because you are subscribed to the Google 
 Groups "Clojure" group.
 To unsubscribe from this group and stop receiving emails from it, send 
 an email to clojure+u...@googlegroups.com.
 For more options, visit https://groups.google.com/d/optout.

>>>
>>>
>>>
>>> -- 
>>> James Reeves
>>> booleanknot.com
>>>
>> -- 
>> You received this message because you are subscribed to the Google
>> Groups "Clojure" group.
>> To post to this group, send email to clo...@googlegroups.com 
>> 
>> Note that posts from new members are moderated - please be patient with 
>> your first post.
>> To unsubscribe from this group, send email to
>> clojure+u...@googlegroups.com 
>> For more options, visit this group at
>> http://groups.google.com/group/clojure?hl=en
>> --- 
>> You received this message because you are subscribed to the Google Groups 
>> "Clojure" group.
>> To unsubscribe from this group and stop receiving emails from it, send an 
>> email to clojure+u...@googlegroups.com .
>> For more options, visit https://groups.google.com/d/optout.
>>
>
>

-- 
You received this message because you are subscribed to the Google
Groups "Clojure" group.
To post to this group, send email to clojure@googlegroups.com
Note that posts from new members are moderated - please be patient with your 
first post.
To unsubscribe from this group, send email to
clojure+unsubscr...@googlegroups.com
For more options, visit this group at
http://groups.google.com/group/clojure?hl=en
--- 
You received this message because you are subscribed to the Google Groups 
"Clojure" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to clojure+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: error in nrepl

[lawrence . krubner]

On Friday, June 23, 2017 at 4:29:24 PM UTC-4, lawrence...@gmail.com wrote:
>
> I'm using Emacs on my Mac. I ran "nrepl-jack-in" to load up the repl. I'm 
> iterating over a dataset from mysql. My code is very simple, I'm just 
> trying to count the words: 
>
> (reduce 
>
> (fn [map-of-word-count next-name] 
> (let [
> words (clojure.string/split next-name #"\s") 
> map-of-names-words-with-count (frequencies words)
> ] 
> (println map-of-names-words-with-count)
> (merge-with + map-of-word-count map-of-names-words-with-count)
> )
> ) 
> {} 
> names)
>
> I keep getting this message:
>
>
> error in process filter: nrepl-bdecode-buffer: Cannot decode object: 1
> error in process filter: Cannot decode object: 1
> Error running timer `jit-lock-stealth-fontify': (error "Variable binding 
> depth exceeds max-specpdl-size")
> timer-relative-time: Variable binding depth exceeds max-specpdl-size
>
>
> Does anyone know what this means? 
>
>

-- 
You received this message because you are subscribed to the Google
Groups "Clojure" group.
To post to this group, send email to clojure@googlegroups.com
Note that posts from new members are moderated - please be patient with your 
first post.
To unsubscribe from this group, send email to
clojure+unsubscr...@googlegroups.com
For more options, visit this group at
http://groups.google.com/group/clojure?hl=en
--- 
You received this message because you are subscribed to the Google Groups 
"Clojure" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to clojure+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: error in nrepl

[lawrence . krubner]

Yes, sadly, I've never gotten Cider to work with Emacs. I keep thinking 
someday I'll take a weekend and work through all the errors and get it 
working, but I never seem to find the time. So I keep working with an old 
version of nrepl. But I take it, from your answer, you think this error 
would vanish if I upgraded to Cider? 




On Friday, June 23, 2017 at 5:15:09 PM UTC-4, James Reeves wrote:
>
> nrepl-jack-in? Do you mean cider-jack-in? AFAIK nrepl-jack-in is from a 
> very old version of Cider.
>
> On 23 June 2017 at 21:29, > wrote:
>
>> I'm using Emacs on my Mac. I ran "nrepl-jack-in" to load up the repl. I'm 
>> iterating over a dataset from mysql. My code is very simple, I'm just 
>> trying to count the words: 
>>
>> (reduce 
>>
>> (fn [map-of-word-count next-name] 
>> (let [
>> words (clojure.string/split next-name #"\s") 
>> map-of-names-words-with-count (frequencies words)
>> ] 
>> (println map-of-names-words-with-count)
>> (merge-with + map-of-word-count map-of-names-words-with-count)
>> )
>> ) 
>> {} 
>> names)
>>
>> I keep getting this message:
>>
>>
>> error in process filter: nrepl-bdecode-buffer: Cannot decode object: 1
>> error in process filter: Cannot decode object: 1
>> Error running timer `jit-lock-stealth-fontify': (error "Variable binding 
>> depth exceeds max-specpdl-size")
>> timer-relative-time: Variable binding depth exceeds max-specpdl-size
>>
>>
>> Does anyone know what this means? 
>>
>> -- 
>> You received this message because you are subscribed to the Google
>> Groups "Clojure" group.
>> To post to this group, send email to clo...@googlegroups.com 
>> 
>> Note that posts from new members are moderated - please be patient with 
>> your first post.
>> To unsubscribe from this group, send email to
>> clojure+u...@googlegroups.com 
>> For more options, visit this group at
>> http://groups.google.com/group/clojure?hl=en
>> --- 
>> You received this message because you are subscribed to the Google Groups 
>> "Clojure" group.
>> To unsubscribe from this group and stop receiving emails from it, send an 
>> email to clojure+u...@googlegroups.com .
>> For more options, visit https://groups.google.com/d/optout.
>>
>
>
>
> -- 
> James Reeves
> booleanknot.com
>

-- 
You received this message because you are subscribed to the Google
Groups "Clojure" group.
To post to this group, send email to clojure@googlegroups.com
Note that posts from new members are moderated - please be patient with your 
first post.
To unsubscribe from this group, send email to
clojure+unsubscr...@googlegroups.com
For more options, visit this group at
http://groups.google.com/group/clojure?hl=en
--- 
You received this message because you are subscribed to the Google Groups 
"Clojure" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to clojure+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

error in nrepl

[lawrence . krubner]

I'm using Emacs on my Mac. I ran "nrepl-jack-in" to load up the repl. I'm 
iterating over a dataset from mysql. My code is very simple, I'm just 
trying to count the words: 

(reduce 

(fn [map-of-word-count next-name] 
(let [
words (clojure.string/split next-name #"\s") 
map-of-names-words-with-count (frequencies words)
] 
(println map-of-names-words-with-count)
(merge-with + map-of-word-count map-of-names-words-with-count)
)
) 
{} 
names)

I keep getting this message:


error in process filter: nrepl-bdecode-buffer: Cannot decode object: 1
error in process filter: Cannot decode object: 1
Error running timer `jit-lock-stealth-fontify': (error "Variable binding 
depth exceeds max-specpdl-size")
timer-relative-time: Variable binding depth exceeds max-specpdl-size


Does anyone know what this means? 

-- 
You received this message because you are subscribed to the Google
Groups "Clojure" group.
To post to this group, send email to clojure@googlegroups.com
Note that posts from new members are moderated - please be patient with your 
first post.
To unsubscribe from this group, send email to
clojure+unsubscr...@googlegroups.com
For more options, visit this group at
http://groups.google.com/group/clojure?hl=en
--- 
You received this message because you are subscribed to the Google Groups 
"Clojure" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to clojure+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: [ANN] meet postagga, a new lib for natural language processing

[lawrence . krubner]

Thank you for this. I am excited to give this a try. 


On Monday, March 13, 2017 at 9:24:36 AM UTC-4, Rafik NACCACHE wrote:
>
> Hey guys,
> I am pleased to share my new lib, capable of training models that help 
> parse natural language !
> My french-speaking friends will find it extremely useful as I did some 
> work to provide some french models!
> Besides, it is pure clojure, no open-nlp whatsoover, so it will happily 
> run on clojure and clojurescript!
> I would be glad if you can give it a shot here: 
> https://github.com/turbopape/postagga
> Cheers !
>
> -- 
>   
>
> [image: --] 
>   
> Rafik Naccache
> [image: https://]about.me/rafik_naccache
>
> 
>  
>

-- 
You received this message because you are subscribed to the Google
Groups "Clojure" group.
To post to this group, send email to clojure@googlegroups.com
Note that posts from new members are moderated - please be patient with your 
first post.
To unsubscribe from this group, send email to
clojure+unsubscr...@googlegroups.com
For more options, visit this group at
http://groups.google.com/group/clojure?hl=en
--- 
You received this message because you are subscribed to the Google Groups 
"Clojure" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to clojure+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: Application silently shuts down by itself after running for some hours

[lawrence . krubner]

To catch OutOfMemoryError s: 

catch(OutOfMemoryError e)




On Tuesday, March 7, 2017 at 5:18:44 PM UTC-5, JokkeB wrote:
>
> I'm under the impression that setDefaultExceptionHandler still catches OOM 
> errors. Some googling suggests this too. If not, how should I try to catch 
> it?
>
> I am running the app on a virtual server with 512mb ram. free -m is 
> showing 30mb free. Lack of memory can be the issue. What would be the best 
> way to confirm this if I'm not able to log the error?
>
> Sent from my iPhone
>
> On 7 Mar 2017, at 23.25, piast...@gmail.com  wrote:
>
> I asked the same question a year ago. The problem was that I was getting 
> an OutOfMemoryError. This is an Error but it is not an Exception. If you 
> catch all Exceptions, you will still not catch the OutOfMemoryError. You 
> have to catch that too. 
>
>
>
> On Tuesday, March 7, 2017 at 2:54:26 PM UTC-5, Kevin Corcoran wrote:
>>
>> On Mon, Mar 6, 2017 at 11:56 PM, JokkeB  wrote:
>>
>>> After adding this, I still can't see an exception before the app dies.
>>>
>>
>> I've encountered this before when the Linux "OOM killer" kicks in, which 
>> is especially likely if you are running your application on a 
>> resource-constrained system (say, a VM with a low RAM allocation) or your 
>> application is competing with other programs for memory.
>>
>> -- 
> You received this message because you are subscribed to the Google
> Groups "Clojure" group.
> To post to this group, send email to clo...@googlegroups.com 
> Note that posts from new members are moderated - please be patient with 
> your first post.
> To unsubscribe from this group, send email to
> clojure+u...@googlegroups.com 
> For more options, visit this group at
> http://groups.google.com/group/clojure?hl=en
> --- 
> You received this message because you are subscribed to a topic in the 
> Google Groups "Clojure" group.
> To unsubscribe from this topic, visit 
> https://groups.google.com/d/topic/clojure/zhdcNMC7fQ8/unsubscribe.
> To unsubscribe from this group and all its topics, send an email to 
> clojure+u...@googlegroups.com .
> For more options, visit https://groups.google.com/d/optout.
>
>

-- 
You received this message because you are subscribed to the Google
Groups "Clojure" group.
To post to this group, send email to clojure@googlegroups.com
Note that posts from new members are moderated - please be patient with your 
first post.
To unsubscribe from this group, send email to
clojure+unsubscr...@googlegroups.com
For more options, visit this group at
http://groups.google.com/group/clojure?hl=en
--- 
You received this message because you are subscribed to the Google Groups 
"Clojure" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to clojure+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: Ensure more concurrency

[lawrence . krubner]

https://clojuredocs.org/clojure.core/ensure

Must be called in a transaction. Protects the ref from modification
by other transactions.  Returns the in-transaction-value of
ref. Allows for more concurrency than (ref-set ref @ref)



This can be read in two contradictory ways. Protecting a ref during a 
transaction sounds like it increases contention and slows the app down. 
"Allows for more concurrency" can be read as "Allows you to use a huge 
number of refs while still ensuring consistency." In other words, it sounds 
like it offers safety at the cost of speed. 




On Monday, March 6, 2017 at 6:06:46 AM UTC-5, bertschi wrote:
>
> For a lecture I have implemented the write-skew example from Mark 
> Volkmann's article:
>
> (defn write-skew []
>   (let [cats (ref 1)
> dogs (ref 1)
>
> john (future
>(dosync
>  (when (< (+ @cats @dogs) 3)
>(alter cats inc
> mary (future
>(dosync
>  (when (< (+ @cats @dogs) 3)
>(alter dogs inc]
> (do @john @mary)
> (dosync
>  (> (+ @cats @dogs) 3
>
> (defn write-skew-demo []
>   (let [count-write-skews (atom 0)]
> (doseq [_ (range 25)]
>   (when (write-skew)
> (swap! count-write-skews inc)))
> @count-write-skews))
>
> (write-skew-demo)
>
> When I try to fix this program using ensure, i.e. using (ensure dogs) in 
> john and (ensure cats) in mary, I find that it sometimes runs very slow.
>
> From the docs it says "Allows for more concurrency than (ref-set ref 
> @ref)". I would read that as "runs at least as fast as (ref-set ref @ref)", 
> but using (ref-set dogs @dogs) instead of (ensure dogs) and the same with 
> cats, does not exhibit these occasional runtime spikes.
>
> Am I misunderstanding something or is it a bug in ensure?
>
>
> Nils
>

-- 
You received this message because you are subscribed to the Google
Groups "Clojure" group.
To post to this group, send email to clojure@googlegroups.com
Note that posts from new members are moderated - please be patient with your 
first post.
To unsubscribe from this group, send email to
clojure+unsubscr...@googlegroups.com
For more options, visit this group at
http://groups.google.com/group/clojure?hl=en
--- 
You received this message because you are subscribed to the Google Groups 
"Clojure" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to clojure+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: If a Java function seems to never return, how do I test?

[Lawrence Krubner]

Possible. I have not dug into the Java app, but I will look for that. 


On Monday, November 2, 2015 at 4:48:38 PM UTC-5, Rob Lally wrote:
>
> Is it possible that the exception is being thrown in a different thread? 
> If that’s the case, you can use:
>
>
> https://docs.oracle.com/javase/7/docs/api/java/lang/Thread.UncaughtExceptionHandler.html
>
> to capture it.
>
>
> Rob.
>
>
> On 2 Nov 2015, at 12:22, Lawrence Krubner  > wrote:
>
>
> Now there is a new error. 
>
> Somehow, when the exception happens in our Java library, even though we, 
> in theory, write to System.out.println(), I never see the Exception in the 
> logs. 
>
> All I can think is that somewhere in the Java library there is an 
> exception that we catch but we forget to do System.out.println(). Can 
> anyone think of another explanation? 
>
>
>
>
> On Sunday, November 1, 2015 at 5:10:19 PM UTC-5, Lawrence Krubner wrote:
>>
>> So, we eventually fixed this. There were 2 bugs that worked together to 
>> make this a mystery. I had to guess at what the problems were, and fix them 
>> blind, since I could not see the Exceptions. I am curious about why I was 
>> not able to see the Exceptions. 
>>
>> About this:
>>
>> > Exceptions are ALWAYS visible, the only way they get lost is 
>> > the try/catch blocks you added in your code which effectively 
>> > swallow and ignore them.
>>
>> But the Exceptions I dealt with were invisible. We have the Clojure app 
>> set up to run via Supervisord and the conf has these lines:
>>
>> stderr_logfile=/var/log/nlph_stderr.log
>> stdout_logfile=/var/log/nlph_stdout.log
>>
>> So in 2 terminal windows I would:
>>
>> cd /var/log
>>
>> tail -f nlph_stdout.log
>>
>> And the same for nlph_stderr.log.
>>
>> So I was looking at the output in my terminal window. And I could see all 
>> of our logging statements appear. And the code clearly got to the line 
>> where the exception happens, but then the Exception never appeared in the 
>> terminal. It was invisible. I assume the output of this: 
>>
>> System.out.println(e.getMessage());
>>
>> would appear in one of the files that I was tailing. But it never did. 
>>
>> So I am wondering why the Exceptions were invisible? Why didn't the print 
>> statements make it to the terminal? 
>>
>>
>>
>>
>> On Saturday, October 31, 2015 at 6:56:23 AM UTC-4, Thomas Heller wrote:
>>>
>>>
>>>> What could we do to make the Exception visible, assuming there is one? 
>>>>
>>>>
>>> Exceptions are ALWAYS visible, the only way they get lost is the 
>>> try/catch blocks you added in your code which effectively swallow and 
>>> ignore them. The JVM will not randomly lose an Exception, it is always code 
>>> you write that decided to handle them in some way. If you do not know how 
>>> to handle an exception do not catch it, printing its stacktrace and 
>>> ignoring it is never a good idea.
>>>
>>> In your case since you say that the function never returns I'd put my 
>>> guess on an infinite loop. There is no exception that gets lost, your code 
>>> is just still running. If your JVM is running locally you can attach to it 
>>> via jvisualvm [1] and get a quick overview of what the JVM is doing. If you 
>>> look at the thread information you'll see if something is still running or 
>>> dead-locked somehow.
>>>
>>> You can also use a debugger and step through the java code step by step.
>>>
>>> Also, consider coding against interfaces in java (eg. java.util.List and 
>>> java.util.Map instead of java.util.Collection), it will save you tons of 
>>> conversion calls.
>>>
>>> HTH
>>> /thomas
>>>
>>> [1] https://visualvm.java.net/
>>>
>>
> -- 
> You received this message because you are subscribed to the Google
> Groups "Clojure" group.
> To post to this group, send email to clo...@googlegroups.com 
> Note that posts from new members are moderated - please be patient with 
> your first post.
> To unsubscribe from this group, send email to
> clojure+u...@googlegroups.com 
> For more options, visit this group at
> http://groups.google.com/group/clojure?hl=en
> --- 
> You received this message because you are subscribed to the Google Groups 
> "Clojure" group.
> To unsubscribe from this group and stop receiving emails from it, send an 
> email to clojure+u...@googlegroups.com .
> 

Re: If a Java function seems to never return, how do I test?

[Lawrence Krubner]

> It is possible to rebind System.out to something else inside Java, so the 
> library could be redirecting it from your view. Not saying that's 
> happening, but it's possible. 

Thanks for this. Now that you mention it, I recall some conversation about 
this over the summer. I will investigate this. 






On Monday, November 2, 2015 at 3:26:15 PM UTC-5, Alex Miller wrote:
>
> It is possible to rebind System.out to something else inside Java, so the 
> library could be redirecting it from your view. Not saying that's 
> happening, but it's possible. 
>
> If you you can connect with a debugger, you can set breakpoints based on 
> any thrown exception.
>
> On Monday, November 2, 2015 at 2:22:30 PM UTC-6, Lawrence Krubner wrote:
>>
>>
>> Now there is a new error. 
>>
>> Somehow, when the exception happens in our Java library, even though we, 
>> in theory, write to System.out.println(), I never see the Exception in the 
>> logs. 
>>
>> All I can think is that somewhere in the Java library there is an 
>> exception that we catch but we forget to do System.out.println(). Can 
>> anyone think of another explanation? 
>>
>>
>>
>>
>> On Sunday, November 1, 2015 at 5:10:19 PM UTC-5, Lawrence Krubner wrote:
>>>
>>> So, we eventually fixed this. There were 2 bugs that worked together to 
>>> make this a mystery. I had to guess at what the problems were, and fix them 
>>> blind, since I could not see the Exceptions. I am curious about why I was 
>>> not able to see the Exceptions. 
>>>
>>> About this:
>>>
>>> > Exceptions are ALWAYS visible, the only way they get lost is 
>>> > the try/catch blocks you added in your code which effectively 
>>> > swallow and ignore them.
>>>
>>> But the Exceptions I dealt with were invisible. We have the Clojure app 
>>> set up to run via Supervisord and the conf has these lines:
>>>
>>> stderr_logfile=/var/log/nlph_stderr.log
>>> stdout_logfile=/var/log/nlph_stdout.log
>>>
>>> So in 2 terminal windows I would:
>>>
>>> cd /var/log
>>>
>>> tail -f nlph_stdout.log
>>>
>>> And the same for nlph_stderr.log.
>>>
>>> So I was looking at the output in my terminal window. And I could see 
>>> all of our logging statements appear. And the code clearly got to the line 
>>> where the exception happens, but then the Exception never appeared in the 
>>> terminal. It was invisible. I assume the output of this: 
>>>
>>> System.out.println(e.getMessage());
>>>
>>> would appear in one of the files that I was tailing. But it never did. 
>>>
>>> So I am wondering why the Exceptions were invisible? Why didn't the 
>>> print statements make it to the terminal? 
>>>
>>>
>>>
>>>
>>> On Saturday, October 31, 2015 at 6:56:23 AM UTC-4, Thomas Heller wrote:
>>>>
>>>>
>>>>> What could we do to make the Exception visible, assuming there is one? 
>>>>>
>>>>>
>>>> Exceptions are ALWAYS visible, the only way they get lost is the 
>>>> try/catch blocks you added in your code which effectively swallow and 
>>>> ignore them. The JVM will not randomly lose an Exception, it is always 
>>>> code 
>>>> you write that decided to handle them in some way. If you do not know how 
>>>> to handle an exception do not catch it, printing its stacktrace and 
>>>> ignoring it is never a good idea.
>>>>
>>>> In your case since you say that the function never returns I'd put my 
>>>> guess on an infinite loop. There is no exception that gets lost, your code 
>>>> is just still running. If your JVM is running locally you can attach to it 
>>>> via jvisualvm [1] and get a quick overview of what the JVM is doing. If 
>>>> you 
>>>> look at the thread information you'll see if something is still running or 
>>>> dead-locked somehow.
>>>>
>>>> You can also use a debugger and step through the java code step by step.
>>>>
>>>> Also, consider coding against interfaces in java (eg. java.util.List 
>>>> and java.util.Map instead of java.util.Collection), it will save you tons 
>>>> of conversion calls.
>>>>
>>>> HTH
>>>> /thomas
>>>>
>>>> [1] https://visualvm.java.net/
>>>>
>>>

-- 
You received this message because you are subscribed to the Google
Groups "Clojure" group.
To post to this group, send email to clojure@googlegroups.com
Note that posts from new members are moderated - please be patient with your 
first post.
To unsubscribe from this group, send email to
clojure+unsubscr...@googlegroups.com
For more options, visit this group at
http://groups.google.com/group/clojure?hl=en
--- 
You received this message because you are subscribed to the Google Groups 
"Clojure" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to clojure+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: If a Java function seems to never return, how do I test?

[Lawrence Krubner]

> flush twice, oracle is far far away? 

Are you saying this is so obvious I should be able to easily look it up? It 
might be obvious to you, but it is not obvious to me. That's why I'm asking.






On Monday, November 2, 2015 at 3:25:36 PM UTC-5, raould wrote:
>
> flush twice, oracle is far far away? 
>

-- 
You received this message because you are subscribed to the Google
Groups "Clojure" group.
To post to this group, send email to clojure@googlegroups.com
Note that posts from new members are moderated - please be patient with your 
first post.
To unsubscribe from this group, send email to
clojure+unsubscr...@googlegroups.com
For more options, visit this group at
http://groups.google.com/group/clojure?hl=en
--- 
You received this message because you are subscribed to the Google Groups 
"Clojure" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to clojure+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: If a Java function seems to never return, how do I test?

[Lawrence Krubner]

Now there is a new error. 

Somehow, when the exception happens in our Java library, even though we, in 
theory, write to System.out.println(), I never see the Exception in the 
logs. 

All I can think is that somewhere in the Java library there is an exception 
that we catch but we forget to do System.out.println(). Can anyone think of 
another explanation? 




On Sunday, November 1, 2015 at 5:10:19 PM UTC-5, Lawrence Krubner wrote:
>
> So, we eventually fixed this. There were 2 bugs that worked together to 
> make this a mystery. I had to guess at what the problems were, and fix them 
> blind, since I could not see the Exceptions. I am curious about why I was 
> not able to see the Exceptions. 
>
> About this:
>
> > Exceptions are ALWAYS visible, the only way they get lost is 
> > the try/catch blocks you added in your code which effectively 
> > swallow and ignore them.
>
> But the Exceptions I dealt with were invisible. We have the Clojure app 
> set up to run via Supervisord and the conf has these lines:
>
> stderr_logfile=/var/log/nlph_stderr.log
> stdout_logfile=/var/log/nlph_stdout.log
>
> So in 2 terminal windows I would:
>
> cd /var/log
>
> tail -f nlph_stdout.log
>
> And the same for nlph_stderr.log.
>
> So I was looking at the output in my terminal window. And I could see all 
> of our logging statements appear. And the code clearly got to the line 
> where the exception happens, but then the Exception never appeared in the 
> terminal. It was invisible. I assume the output of this: 
>
> System.out.println(e.getMessage());
>
> would appear in one of the files that I was tailing. But it never did. 
>
> So I am wondering why the Exceptions were invisible? Why didn't the print 
> statements make it to the terminal? 
>
>
>
>
> On Saturday, October 31, 2015 at 6:56:23 AM UTC-4, Thomas Heller wrote:
>>
>>
>>> What could we do to make the Exception visible, assuming there is one? 
>>>
>>>
>> Exceptions are ALWAYS visible, the only way they get lost is the 
>> try/catch blocks you added in your code which effectively swallow and 
>> ignore them. The JVM will not randomly lose an Exception, it is always code 
>> you write that decided to handle them in some way. If you do not know how 
>> to handle an exception do not catch it, printing its stacktrace and 
>> ignoring it is never a good idea.
>>
>> In your case since you say that the function never returns I'd put my 
>> guess on an infinite loop. There is no exception that gets lost, your code 
>> is just still running. If your JVM is running locally you can attach to it 
>> via jvisualvm [1] and get a quick overview of what the JVM is doing. If you 
>> look at the thread information you'll see if something is still running or 
>> dead-locked somehow.
>>
>> You can also use a debugger and step through the java code step by step.
>>
>> Also, consider coding against interfaces in java (eg. java.util.List and 
>> java.util.Map instead of java.util.Collection), it will save you tons of 
>> conversion calls.
>>
>> HTH
>> /thomas
>>
>> [1] https://visualvm.java.net/
>>
>

-- 
You received this message because you are subscribed to the Google
Groups "Clojure" group.
To post to this group, send email to clojure@googlegroups.com
Note that posts from new members are moderated - please be patient with your 
first post.
To unsubscribe from this group, send email to
clojure+unsubscr...@googlegroups.com
For more options, visit this group at
http://groups.google.com/group/clojure?hl=en
--- 
You received this message because you are subscribed to the Google Groups 
"Clojure" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to clojure+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: If a Java function seems to never return, how do I test?

[Lawrence Krubner]

So, we eventually fixed this. There were 2 bugs that worked together to 
make this a mystery. I had to guess at what the problems were, and fix them 
blind, since I could not see the Exceptions. I am curious about why I was 
not able to see the Exceptions. 

About this:

> Exceptions are ALWAYS visible, the only way they get lost is 
> the try/catch blocks you added in your code which effectively 
> swallow and ignore them.

But the Exceptions I dealt with were invisible. We have the Clojure app set 
up to run via Supervisord and the conf has these lines:

stderr_logfile=/var/log/nlph_stderr.log
stdout_logfile=/var/log/nlph_stdout.log

So in 2 terminal windows I would:

cd /var/log

tail -f nlph_stdout.log

And the same for nlph_stderr.log.

So I was looking at the output in my terminal window. And I could see all 
of our logging statements appear. And the code clearly got to the line 
where the exception happens, but then the Exception never appeared in the 
terminal. It was invisible. I assume the output of this: 

System.out.println(e.getMessage());

would appear in one of the files that I was tailing. But it never did. 

So I am wondering why the Exceptions were invisible? Why didn't the print 
statements make it to the terminal? 




On Saturday, October 31, 2015 at 6:56:23 AM UTC-4, Thomas Heller wrote:
>
>
>> What could we do to make the Exception visible, assuming there is one? 
>>
>>
> Exceptions are ALWAYS visible, the only way they get lost is the try/catch 
> blocks you added in your code which effectively swallow and ignore them. 
> The JVM will not randomly lose an Exception, it is always code you write 
> that decided to handle them in some way. If you do not know how to handle 
> an exception do not catch it, printing its stacktrace and ignoring it is 
> never a good idea.
>
> In your case since you say that the function never returns I'd put my 
> guess on an infinite loop. There is no exception that gets lost, your code 
> is just still running. If your JVM is running locally you can attach to it 
> via jvisualvm [1] and get a quick overview of what the JVM is doing. If you 
> look at the thread information you'll see if something is still running or 
> dead-locked somehow.
>
> You can also use a debugger and step through the java code step by step.
>
> Also, consider coding against interfaces in java (eg. java.util.List and 
> java.util.Map instead of java.util.Collection), it will save you tons of 
> conversion calls.
>
> HTH
> /thomas
>
> [1] https://visualvm.java.net/
>

-- 
You received this message because you are subscribed to the Google
Groups "Clojure" group.
To post to this group, send email to clojure@googlegroups.com
Note that posts from new members are moderated - please be patient with your 
first post.
To unsubscribe from this group, send email to
clojure+unsubscr...@googlegroups.com
For more options, visit this group at
http://groups.google.com/group/clojure?hl=en
--- 
You received this message because you are subscribed to the Google Groups 
"Clojure" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to clojure+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

If a Java function seems to never return, how do I test?

[Lawrence Krubner]

I am trying to figure out where this code might die, and why we don't se 
the Exception, if there is an Exception. 

Inside of a try/catch block, we have this, when calls a library written in 
Java: 


(timbre/log :trace " in parse-sentence fields-and-labels is: " 
fields-and-labels)

(let [
  instance-of-sentence-parser @nlp-engine
  sentence-parse-response-map (.init instance-of-sentence-parser
 (:incoming-message 
this-users-conversation)
 organization
 accounts
 contacts
 required-fields
 fields-and-labels)
  ]

(timbre/log :trace " in parse-sentence sentence-parse-response-map is: " 
sentence-parse-response-map)


We get to the first of these log statements, but not the second. The Java 
function we call looks like this: 


public HashMap init(String debrief,String companyName, 
Collection contacts, Collection accounts, Collection requiredFields, Map 
fieldLabelMap) {
try {
System.out.println("The house is inside the nlp init.");
ArrayList ct = new 
ArrayList(Arrays.asList(contacts.toArray(new String[0])));
ArrayList ac = new 
ArrayList(Arrays.asList(accounts.toArray(new String[0])));
ArrayList rq = new 
ArrayList(Arrays.asList(requiredFields.toArray(new String[0])));
HashMap flMap = new HashMap<>(fieldLabelMap);
System.out.println("init method has ended.");
return transformer.transform(debrief, companyName, 
tecClassifier, rollioClassifier, caseClassifier, caselessClassifier, 
customClassifier, pipeline, parser, props, firstNames, lastNames, ac, ct, 
rq, flMap);
} catch (Exception e) {
System.out.println("Top level Exception in Main::main" + 
e.getMessage());
e.printStackTrace();
}
return null;
}


And it gets as far as :

System.out.println("init method has ended.");

The transform function looks like: 

public HashMap transform(String debrief, String 
companyName, AbstractSequenceClassifier tecClassifier, 
AbstractSequenceClassifier 
amaranClassifier,AbstractSequenceClassifier 
caseClassifier,AbstractSequenceClassifier caselessClassifier, 
AbstractSequenceClassifier customClassifier,StanfordCoreNLP 
pipeline, Parser parser, Properties props,String[] firstNames, String[] 
lastNames, ArrayList acctMp, ArrayList contactMap, 
ArrayList requiredFields, HashMap fieldLabelMap) {
try {
if (companyName.equals("Amaran")) {
AmaranSFInfoExtractor sfInfoExtractor = new 
AmaranSFInfoExtractor(debrief, pipeline, parser, amaranClassifier, 
caseClassifier, caselessClassifier, customClassifier, props, firstNames, 
lastNames, acctMp, contactMap, requiredFields, fieldLabelMap);
sfInfoExtractor.extract();
return deNestMap(sfInfoExtractor.tagMap);
} if (companyName.equals("TEC")) {
TECSFInfoExtractor sfInfoExtractor = new 
TECSFInfoExtractor(debrief, pipeline, parser, tecClassifier, 
caseClassifier, caselessClassifier, customClassifier, props, firstNames, 
lastNames, acctMp, contactMap, requiredFields, fieldLabelMap);
sfInfoExtractor.extract();
return deNestMap(sfInfoExtractor.tecTagMap);
}
} catch (Exception e) {
System.out.println(e.getMessage());
e.printStackTrace();
}
return new HashMap<>();
}


Where does this die? And why don't we see the Exception? 

What could we do to make the Exception visible, assuming there is one? 





-- 
You received this message because you are subscribed to the Google
Groups "Clojure" group.
To post to this group, send email to clojure@googlegroups.com
Note that posts from new members are moderated - please be patient with your 
first post.
To unsubscribe from this group, send email to
clojure+unsubscr...@googlegroups.com
For more options, visit this group at
http://groups.google.com/group/clojure?hl=en
--- 
You received this message because you are subscribed to the Google Groups 
"Clojure" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to clojure+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

is there a community "best practice" for including your co-workers libraries?

[Lawrence Krubner]

I know this question has been asked before, but when I went searching I 
mostly found old entries that were on a somewhat different topic, such as 
"How to add a java library (that is not in maven) as a dependency for a 
clojure library?"

I wrote a Clojure app and my co-worker wrote a Java app. I have another 
co-worker who is working on an app that would include the first 2 apps as 
libraries. 

I have been able to get the combination working using the Leiningen plugin 
"localrepo" but my co-worker wants me to make the other 2 libraries 
available from some central repo that Leiningen can download from. How is 
this usually done? Do I set up a Maven repo on one of the company servers? 








-- 
You received this message because you are subscribed to the Google
Groups "Clojure" group.
To post to this group, send email to clojure@googlegroups.com
Note that posts from new members are moderated - please be patient with your 
first post.
To unsubscribe from this group, send email to
clojure+unsubscr...@googlegroups.com
For more options, visit this group at
http://groups.google.com/group/clojure?hl=en
--- 
You received this message because you are subscribed to the Google Groups 
"Clojure" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to clojure+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: "get" returns nil even when "not-found" is supplied?

[Lawrence Krubner]

> Is it possible that this-users-converstation *does* have 
> an :incoming-message key with a nil value?

Good catch! I feel like an idiot for not realizing that. 



On Friday, October 16, 2015 at 2:43:14 PM UTC-4, Francis Avila wrote:
>
> Not-found is not the same is not-nil. Is it possible that 
> this-users-converstation *does* have an :incoming-message key with a nil 
> value?
>
> See the difference between these two cases:
>
> (get {} :a :not-found)
> => :not-found
> (get {:a nil} :a :not-found)
> => nil
>
>
>
> Notice also that records always "have" keys that are defined on them even 
> if they are not set:
>
> (defrecord Rec [a])=> user.Rec
> (get (map->Rec {}) :a :not-found)
> => nil
>
>
>
> Perhaps you want to use or instead?
>
> (or (get this-users-conversation :incoming-message) "")
>
>
> Or figure out why nil is written and prevent it?
>
> On Friday, October 16, 2015 at 1:33:29 PM UTC-5, Lawrence Krubner wrote:
>>
>> What am I doing wrong here? I want to call clojure.string/lower-case  on 
>> the :incoming-message of this-users-conversation. If there is no message, I 
>> return an empty string. 
>>
>>
>> (defn discern-current-state [this-users-conversation]
>>   (cond
>> (= (clojure.string/lower-case (get this-users-conversation 
>> :incoming-message "")) "yes") (assoc this-users-conversation :current-state 
>> :salesforce-write)
>>
>>
>> and yet the error points to the above line: 
>>
>> java.lang.NullPointerException {:class java.lang.NullPointerException, 
>> :message nil, :trace-elems ({:anon-fn false, :fn "lower-case", :ns 
>> "clojure.string", :clojure true, :file "string.clj", :line 215} {:anon-fn 
>> false, :fn "discern-current-state", :ns "nlph.event-bus", :clojure true, 
>> :file "event_bus.clj", :line 92} 
>>
>>
>> but of course, at the REPL, everything works as I would expect: 
>>
>> (def users {:message "hello"})
>> #'nlph.core/users
>>
>> nlph.core=> (clojure.string/lower-case (get users :message))
>> "hello"
>>
>> nlph.core=> (clojure.string/lower-case (get users :message ""))
>> "hello"
>>
>> nlph.core=> (clojure.string/lower-case (get users :lisa))
>>
>> NullPointerException   clojure.string/lower-case (string.clj:215)
>>
>> nlph.core=> (clojure.string/lower-case (get users :lisa ""))
>> ""
>>
>> nlph.core=> (clojure.string/lower-case (get users :lisa))
>>
>> NullPointerException   clojure.string/lower-case (string.clj:215)
>>
>

-- 
You received this message because you are subscribed to the Google
Groups "Clojure" group.
To post to this group, send email to clojure@googlegroups.com
Note that posts from new members are moderated - please be patient with your 
first post.
To unsubscribe from this group, send email to
clojure+unsubscr...@googlegroups.com
For more options, visit this group at
http://groups.google.com/group/clojure?hl=en
--- 
You received this message because you are subscribed to the Google Groups 
"Clojure" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to clojure+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

"get" returns nil even when "not-found" is supplied?

[Lawrence Krubner]

What am I doing wrong here? I want to call clojure.string/lower-case  on 
the :incoming-message of this-users-conversation. If there is no message, I 
return an empty string. 


(defn discern-current-state [this-users-conversation]
  (cond
(= (clojure.string/lower-case (get this-users-conversation 
:incoming-message "")) "yes") (assoc this-users-conversation :current-state 
:salesforce-write)


and yet the error points to the above line: 

java.lang.NullPointerException {:class java.lang.NullPointerException, 
:message nil, :trace-elems ({:anon-fn false, :fn "lower-case", :ns 
"clojure.string", :clojure true, :file "string.clj", :line 215} {:anon-fn 
false, :fn "discern-current-state", :ns "nlph.event-bus", :clojure true, 
:file "event_bus.clj", :line 92} 


but of course, at the REPL, everything works as I would expect: 

(def users {:message "hello"})
#'nlph.core/users

nlph.core=> (clojure.string/lower-case (get users :message))
"hello"

nlph.core=> (clojure.string/lower-case (get users :message ""))
"hello"

nlph.core=> (clojure.string/lower-case (get users :lisa))

NullPointerException   clojure.string/lower-case (string.clj:215)

nlph.core=> (clojure.string/lower-case (get users :lisa ""))
""

nlph.core=> (clojure.string/lower-case (get users :lisa))

NullPointerException   clojure.string/lower-case (string.clj:215)

-- 
You received this message because you are subscribed to the Google
Groups "Clojure" group.
To post to this group, send email to clojure@googlegroups.com
Note that posts from new members are moderated - please be patient with your 
first post.
To unsubscribe from this group, send email to
clojure+unsubscr...@googlegroups.com
For more options, visit this group at
http://groups.google.com/group/clojure?hl=en
--- 
You received this message because you are subscribed to the Google Groups 
"Clojure" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to clojure+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: what is the shortest series of casts needed to get

[Lawrence Krubner]

Thank you. This was a good idea: 

> So if your coworkers code just needs something iterable (for example), 
you may 

> not need to do any casting at all: just give him the vector.




On Sunday, October 4, 2015 at 9:39:36 PM UTC-4, Francis Avila wrote:
>
> Does he actually need a real arraylist, or will something fulfilling a 
> collection interface (Collection, Iterable, or List for example) be ok? 
> Many clojure types do not require any casting at all as long as the java 
> code writes to a collection interface and doesn't expect to be able to 
> mutate the object. (Both of these are good Java idioms: write to interfaces 
> not classes, and copy defensively.)
>
> So if your coworkers code just needs something iterable (for example), you 
> may not need to do any casting at all: just give him the vector.
>
> If your coworker's code needs a real ArrayList and can accept no 
> substitutes, (java.util.ArrayList. your-vector) will make an ArrayList copy 
> of a vector's contents.
>
>

-- 
You received this message because you are subscribed to the Google
Groups "Clojure" group.
To post to this group, send email to clojure@googlegroups.com
Note that posts from new members are moderated - please be patient with your 
first post.
To unsubscribe from this group, send email to
clojure+unsubscr...@googlegroups.com
For more options, visit this group at
http://groups.google.com/group/clojure?hl=en
--- 
You received this message because you are subscribed to the Google Groups 
"Clojure" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to clojure+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: How does one print the stacktrace from slingshot?

[Lawrence Krubner]

Thank you.


On Friday, October 9, 2015 at 6:16:23 AM UTC-4, Marc O'Morain wrote:
>
> If you pass (:throwable &throw-context) as an arg before the string it 
> will print the stack trace:
>
> (timbre/log :trace (:throwable &throw-context)  "the true stack trace: ") 
>
>
>
>
> On Fri, Oct 9, 2015 at 11:00 AM, Lawrence Krubner  > wrote:
>
>>
>> I'm trying to use Slingshot. I would like to print the stacktrace. If I 
>> do: 
>>
>>  (timbre/log :trace " the stack trace via slingshot-support " 
>> (slingshot-support/stack-trace))
>>
>>
>> I get: 
>>
>> the stack trace via slingshot-support 
>>  [Ljava.lang.StackTraceElement;@72ee66cd
>>
>> How do I get what is in StackTraceElement? 
>>
>> I looked at these articles: 
>>
>> http://www.tutorialspoint.com/java/lang/stacktraceelement_tostring.htm
>>
>>
>> http://www.javaworld.com/article/2072391/the-surprisingly-simple-stacktraceelement.html
>>
>> but I could not figure out how to translate that to what I'm doing. 
>>
>> I also tried: 
>>
>>  (timbre/log :trace " the true stack trace: "(.getStackTrace 
>> (:throwable &throw-context)))
>>
>> but that gives me: 
>>
>>  the true stack trace:  [Ljava.lang.StackTraceElement;@3733df3
>>
>> I can not figure out how to see what is in StackTraceElement. 
>>
>> If I search on Google for "slingshot print stacktrace" I can not find any 
>> examples: 
>>
>>
>> https://www.google.com/search?q=slingshot+print+stacktrace&oq=slingshot+pr&aqs=chrome.0.69i59j0j69i57j0l3.3861j0j7&sourceid=chrome&es_sm=91&ie=UTF-8
>>
>>
>>
>>
>>
>> -- 
>> You received this message because you are subscribed to the Google
>> Groups "Clojure" group.
>> To post to this group, send email to clo...@googlegroups.com 
>> 
>> Note that posts from new members are moderated - please be patient with 
>> your first post.
>> To unsubscribe from this group, send email to
>> clojure+u...@googlegroups.com 
>> For more options, visit this group at
>> http://groups.google.com/group/clojure?hl=en
>> --- 
>> You received this message because you are subscribed to the Google Groups 
>> "Clojure" group.
>> To unsubscribe from this group and stop receiving emails from it, send an 
>> email to clojure+u...@googlegroups.com .
>> For more options, visit https://groups.google.com/d/optout.
>>
>
>

-- 
You received this message because you are subscribed to the Google
Groups "Clojure" group.
To post to this group, send email to clojure@googlegroups.com
Note that posts from new members are moderated - please be patient with your 
first post.
To unsubscribe from this group, send email to
clojure+unsubscr...@googlegroups.com
For more options, visit this group at
http://groups.google.com/group/clojure?hl=en
--- 
You received this message because you are subscribed to the Google Groups 
"Clojure" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to clojure+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: How does one print the stacktrace from slingshot?

[Lawrence Krubner]

Ah, not enough sleep. I was catching it a lower level and re throwing it in 
a form that tripped me up. Got rid of low level catch and did: 

 (timbre/log :trace " the true stack trace: " 
(stack/parse-exception (:throwable &throw-context)))

using clj-stacktrace and all is good. 


On Friday, October 9, 2015 at 6:00:35 AM UTC-4, Lawrence Krubner wrote:
>
>
> I'm trying to use Slingshot. I would like to print the stacktrace. If I 
> do: 
>
>  (timbre/log :trace " the stack trace via slingshot-support " 
> (slingshot-support/stack-trace))
>
>
> I get: 
>
> the stack trace via slingshot-support 
>  [Ljava.lang.StackTraceElement;@72ee66cd
>
> How do I get what is in StackTraceElement? 
>
> I looked at these articles: 
>
> http://www.tutorialspoint.com/java/lang/stacktraceelement_tostring.htm
>
>
> http://www.javaworld.com/article/2072391/the-surprisingly-simple-stacktraceelement.html
>
> but I could not figure out how to translate that to what I'm doing. 
>
> I also tried: 
>
>  (timbre/log :trace " the true stack trace: "(.getStackTrace 
> (:throwable &throw-context)))
>
> but that gives me: 
>
>  the true stack trace:  [Ljava.lang.StackTraceElement;@3733df3
>
> I can not figure out how to see what is in StackTraceElement. 
>
> If I search on Google for "slingshot print stacktrace" I can not find any 
> examples: 
>
>
> https://www.google.com/search?q=slingshot+print+stacktrace&oq=slingshot+pr&aqs=chrome.0.69i59j0j69i57j0l3.3861j0j7&sourceid=chrome&es_sm=91&ie=UTF-8
>
>
>
>
>
>

-- 
You received this message because you are subscribed to the Google
Groups "Clojure" group.
To post to this group, send email to clojure@googlegroups.com
Note that posts from new members are moderated - please be patient with your 
first post.
To unsubscribe from this group, send email to
clojure+unsubscr...@googlegroups.com
For more options, visit this group at
http://groups.google.com/group/clojure?hl=en
--- 
You received this message because you are subscribed to the Google Groups 
"Clojure" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to clojure+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.