Map holds on to element being processed
Ran into an interesting problem today. In short, this works: (count (repeat 1e8 "stuff")) But this doesn't: (map count [(repeat 1e8 "stuff")]) To be fair, given sufficient memory, it would eventually complete. (If the second example does work for you, change it to 1e10 or something higher). The first one works because nothing is holding on to the head of the seq. My assumption is that the second is eating memory because map still has a reference to the item being processed, while the call to count is causing it to be evaluated. Thus the whole seq is retained and we run out of memory. Is my guess correct? If so, is there a workaround for this? Thanks, Matt -- You received this message because you are subscribed to the Google Groups "Clojure" group. To post to this group, send email to clojure@googlegroups.com Note that posts from new members are moderated - please be patient with your first post. To unsubscribe from this group, send email to clojure+unsubscr...@googlegroups.com For more options, visit this group at http://groups.google.com/group/clojure?hl=en --- You received this message because you are subscribed to the Google Groups "Clojure" group. To unsubscribe from this group and stop receiving emails from it, send an email to clojure+unsubscr...@googlegroups.com. For more options, visit https://groups.google.com/d/optout.
Re: Map holds on to element being processed
Thanks! I was playing around with similar variants last night & came up with some that seem to work for one element, but not many. I was seeing a similar result from your version: => (map2 count [(repeat 1e8 "stuff") (repeat 1e8 "stuff") (repeat 1e8 "stuff")]) OutOfMemoryError GC overhead limit exceeded java.lang.Double.valueOf (Double.java:521) clojure.lang.Numbers$DoubleOps.dec (Numbers.java:628) clojure.lang.Numbers.dec (Numbers.java:118) clojure.core/take/fn--4270 (core.clj:2627) clojure.lang.LazySeq.sval (LazySeq.java:40) clojure.lang.LazySeq.seq (LazySeq.java:49) clojure.lang.Cons.next (Cons.java:39) clojure.lang.RT.countFrom (RT.java:540) clojure.lang.RT.count (RT.java:530) clojure.core/count (core.clj:839) pigpen.runtime/map2/fn--1344 (NO_SOURCE_FILE:5) clojure.lang.LazySeq.sval (LazySeq.java:40) But then it dawned on us to try a list instead of a vector for the main seq: => (map2 count (list (repeat 1e8 "stuff") (repeat 1e8 "stuff") (repeat 1e8 "stuff"))) (1 1 1) It does end up thrashing with GCs a bit, but in the end it does the job. It seems like the vector was holding on to something that the list doesn't. It's unfortunate that I would need a custom map implementation to make this work. Elsewhere in the code I (somewhat accidentally) ended up using async channels to solve the same problem. Instead of modeling the large collection as a lazy seq, I have a producer put items into a collection and a consumer read from the collection and transform them (count in this example). In general, would it be better to use channels instead of lazy seqs for very large sequences? Lazy seqs seem to have a couple of disadvantages at scale: you have to be really careful not to hold on to the head (frequently this is hidden anyway), and the evaluation of the lazy seq seems to stress out the GC. Are there other alternatives for large seqs that are better than either of these options? Thanks, Matt On Monday, November 10, 2014 at 10:47 PM, Andy Fingerhut wrote: > At least in your particular case, replacing map with map2, defined below as a > small modification to a subset of map, seems to do the trick: > > (defn map2 [f coll] > (lazy-seq >(when-let [s (seq coll)] > (let [r (rest s)] >(cons (f (first s)) (map2 f r)) > > > (map2 count [(repeat 1e8 "stuff")]) > > I believe this is because the original definition of map, or the subset of it > below: > > (defn map [f coll] > (lazy-seq >(when-let [s (seq coll)] > (cons (f (first s)) (map f (rest s)) > > > holds onto the head via needing to keep the value of "s" around throughout > the entire call to (f (first s)) in order to later make the call (map f (rest > s)). In map2, the value of s is no longer needed by the time f is called. > > Andy > > On Mon, Nov 10, 2014 at 7:48 PM, 'Matt Bossenbroek' via Clojure > mailto:clojure@googlegroups.com)> wrote: > > Ran into an interesting problem today. In short, this works: > > > > (count (repeat 1e8 "stuff")) > > > > But this doesn't: > > > > (map count [(repeat 1e8 "stuff")]) > > > > To be fair, given sufficient memory, it would eventually complete. (If the > > second example does work for you, change it to 1e10 or something higher). > > > > The first one works because nothing is holding on to the head of the seq. > > My assumption is that the second is eating memory because map still has a > > reference to the item being processed, while the call to count is causing > > it to be evaluated. Thus the whole seq is retained and we run out of memory. > > > > Is my guess correct? If so, is there a workaround for this? > > > > Thanks, > > Matt > > > > -- > > You received this message because you are subscribed to the Google > > Groups "Clojure" group. > > To post to this group, send email to clojure@googlegroups.com > > (mailto:clojure@googlegroups.com) > > Note that posts from new members are moderated - please be patient with > > your first post. > > To unsubscribe from this group, send email to > > clojure+unsubscr...@googlegroups.com > > (mailto:clojure%2bunsubscr...@googlegroups.com) > > For more options, visit this group at > > http://groups.google.com/group/clojure?hl=en > > --- > > You received this message because you are subscribed to the Google Groups > > "Clojure" group. > > To unsubscribe from this group and stop receiving emails from it, send an > > email to clojure+unsubscr...@googlegroups.com > > (mailto:clojure+u
Re: pigpen newbie question
Just saw this response - disregard the questions I asked you on the pigpen support DL. I'll pull in the new instaparse & get a new PigPen build out soonish (within a day or two). -Matt On Thursday, September 11, 2014 at 6:28 PM, Mark Engelberg wrote: > You're probably using Clojure 1.7.0 alpha 2, which introduced a new function > called "cat" into the core namespace, which overlaps with a function in > instaparse. > > A couple nights ago, I updated instaparse to version 1.3.4, with an update to > deal with this change in alpha 2, but pigpen has not yet been updated to use > that version of instaparse. > > You can either go back to a non-alpha release of Clojure, or wait for the > pigpen folks to update, or perhaps there is some leiningen-fu you can do in > the project.clj file to override the instaparse dependency loaded by pigpen > with instaparse 1.3.4. > > On Thu, Sep 11, 2014 at 5:16 PM, Sunil S Nandihalli > mailto:sunil.nandiha...@gmail.com)> wrote: > > Hi , > > I am trying to compile a simple clj file which does nothing apart from > > requiring the pigpen name-space and it fails to compile with the following > > error. Can anybody help? > > > > Attempting to call unbound fn: #'instaparse.combinators-source/cat > > > > the full stack trace is here. > > https://gist.github.com/sunilnandihalli/b400e21552ca97038e56 > > > > Thanks, > > Sunil. > > > > > > -- > > You received this message because you are subscribed to the Google > > Groups "Clojure" group. > > To post to this group, send email to clojure@googlegroups.com > > (mailto:clojure@googlegroups.com) > > Note that posts from new members are moderated - please be patient with > > your first post. > > To unsubscribe from this group, send email to > > clojure+unsubscr...@googlegroups.com > > (mailto:clojure%2bunsubscr...@googlegroups.com) > > For more options, visit this group at > > http://groups.google.com/group/clojure?hl=en > > --- > > You received this message because you are subscribed to the Google Groups > > "Clojure" group. > > To unsubscribe from this group and stop receiving emails from it, send an > > email to clojure+unsubscr...@googlegroups.com > > (mailto:clojure+unsubscr...@googlegroups.com). > > For more options, visit https://groups.google.com/d/optout. > > -- > You received this message because you are subscribed to the Google > Groups "Clojure" group. > To post to this group, send email to clojure@googlegroups.com > (mailto:clojure@googlegroups.com) > Note that posts from new members are moderated - please be patient with your > first post. > To unsubscribe from this group, send email to > clojure+unsubscr...@googlegroups.com > (mailto:clojure+unsubscr...@googlegroups.com) > For more options, visit this group at > http://groups.google.com/group/clojure?hl=en > --- > You received this message because you are subscribed to the Google Groups > "Clojure" group. > To unsubscribe from this group and stop receiving emails from it, send an > email to clojure+unsubscr...@googlegroups.com > (mailto:clojure+unsubscr...@googlegroups.com). > For more options, visit https://groups.google.com/d/optout. -- You received this message because you are subscribed to the Google Groups "Clojure" group. To post to this group, send email to clojure@googlegroups.com Note that posts from new members are moderated - please be patient with your first post. To unsubscribe from this group, send email to clojure+unsubscr...@googlegroups.com For more options, visit this group at http://groups.google.com/group/clojure?hl=en --- You received this message because you are subscribed to the Google Groups "Clojure" group. To unsubscribe from this group and stop receiving emails from it, send an email to clojure+unsubscr...@googlegroups.com. For more options, visit https://groups.google.com/d/optout.
Re: pigpen newbie question
Sunil, I tried upgrading PigPen to Instaparse 1.3.4, but that pulled in Clojure 1.6.0 & now I'm running into some build/jar/versioning issues. I don't think I'll be able to get the update out as soon as promised, but it sounds like not using 1.7.0 will work for you in the meantime. -Matt On Thursday, September 11, 2014 at 7:52 PM, Sunil S Nandihalli wrote: > Thanks Mark and Matt, changing the version back to clojure version 1.6.0 > fixed it. > Sunil > > > On Fri, Sep 12, 2014 at 7:05 AM, 'Matt Bossenbroek' via Clojure > mailto:clojure@googlegroups.com)> wrote: > > Just saw this response - disregard the questions I asked you on the pigpen > > support DL. > > > > I'll pull in the new instaparse & get a new PigPen build out soonish > > (within a day or two). > > > > -Matt > > > > > > On Thursday, September 11, 2014 at 6:28 PM, Mark Engelberg wrote: > > > > > You're probably using Clojure 1.7.0 alpha 2, which introduced a new > > > function called "cat" into the core namespace, which overlaps with a > > > function in instaparse. > > > > > > A couple nights ago, I updated instaparse to version 1.3.4, with an > > > update to deal with this change in alpha 2, but pigpen has not yet been > > > updated to use that version of instaparse. > > > > > > You can either go back to a non-alpha release of Clojure, or wait for the > > > pigpen folks to update, or perhaps there is some leiningen-fu you can do > > > in the project.clj file to override the instaparse dependency loaded by > > > pigpen with instaparse 1.3.4. > > > > > > On Thu, Sep 11, 2014 at 5:16 PM, Sunil S Nandihalli > > > mailto:sunil.nandiha...@gmail.com)> wrote: > > > > Hi , > > > > I am trying to compile a simple clj file which does nothing apart from > > > > requiring the pigpen name-space and it fails to compile with the > > > > following error. Can anybody help? > > > > > > > > Attempting to call unbound fn: #'instaparse.combinators-source/cat > > > > > > > > the full stack trace is here. > > > > https://gist.github.com/sunilnandihalli/b400e21552ca97038e56 > > > > > > > > Thanks, > > > > Sunil. > > > > > > > > > > > > -- > > > > You received this message because you are subscribed to the Google > > > > Groups "Clojure" group. > > > > To post to this group, send email to clojure@googlegroups.com > > > > (mailto:clojure@googlegroups.com) > > > > Note that posts from new members are moderated - please be patient with > > > > your first post. > > > > To unsubscribe from this group, send email to > > > > clojure+unsubscr...@googlegroups.com > > > > (mailto:clojure%2bunsubscr...@googlegroups.com) > > > > For more options, visit this group at > > > > http://groups.google.com/group/clojure?hl=en > > > > --- > > > > You received this message because you are subscribed to the Google > > > > Groups "Clojure" group. > > > > To unsubscribe from this group and stop receiving emails from it, send > > > > an email to clojure+unsubscr...@googlegroups.com > > > > (mailto:clojure+unsubscr...@googlegroups.com). > > > > For more options, visit https://groups.google.com/d/optout. > > > > > > -- > > > You received this message because you are subscribed to the Google > > > Groups "Clojure" group. > > > To post to this group, send email to clojure@googlegroups.com > > > (mailto:clojure@googlegroups.com) > > > Note that posts from new members are moderated - please be patient with > > > your first post. > > > To unsubscribe from this group, send email to > > > clojure+unsubscr...@googlegroups.com > > > (mailto:clojure+unsubscr...@googlegroups.com) > > > For more options, visit this group at > > > http://groups.google.com/group/clojure?hl=en > > > --- > > > You received this message because you are subscribed to the Google Groups > > > "Clojure" group. > > > To unsubscribe from this group and stop receiving emails from it, send an > > > email to clojure+unsubscr...@googlegroups.com > > > (mailto:clojure+unsubscr...@googlegroups.com). > > > For more options, visit https://groups.google.com/d/optout. > > > > -- >
Re: [ANN] PigPen 0.3.0 - Now with Cascading!
No complaints, so PigPen 0.3.0 is now officially released. Enjoy! -Matt On Monday, May 11, 2015 at 8:40 AM, Matt Bossenbroek wrote: > I'm excited to announce the release of PigPen v0.3.0, which now includes > support for Cascading. > > PigPen is Map-Reduce for Clojure - you write idiomatic Clojure code, we > compile it into an Apache Pig script or a Cascading flow that runs on Hadoop. > > https://github.com/Netflix/PigPen > > An RC build is currently available in Maven (0.3.0-rc.7). We've been using it > at Netflix for a little over a month now with no issues, but we want to get > it out in the wild before locking down the release. > > There are a couple of breaking changes in this release, mostly just moving > pig-specific functions into another namespace. Check out the release notes > for more details. > > Also new since the last major release: semi-joins, anti-joins, load-json, > store-json, load-csv (RFC 4180), load-parquet, store-parquet, and load-avro > (parquet & avro support not yet available for cascading). > > A huge thanks to Piotr Kozikowski for contributing all of the Cascading work > to the project. > > For questions or complaints: pigpen-supp...@googlegroups.com > (mailto:pigpen-supp...@googlegroups.com) > > -Matt > > > ps. The PigPen name & logo aren't really ideal now that we've added another > platform and plan to add more. Apparently we're only clever enough for one > name & logo. If you've got an idea for a better name for the project, please > let us know. :) > > -- > You received this message because you are subscribed to the Google > Groups "Clojure" group. > To post to this group, send email to clojure@googlegroups.com > (mailto:clojure@googlegroups.com) > Note that posts from new members are moderated - please be patient with your > first post. > To unsubscribe from this group, send email to > clojure+unsubscr...@googlegroups.com > (mailto:clojure+unsubscr...@googlegroups.com) > For more options, visit this group at > http://groups.google.com/group/clojure?hl=en > --- > You received this message because you are subscribed to the Google Groups > "Clojure" group. > To unsubscribe from this group and stop receiving emails from it, send an > email to clojure+unsubscr...@googlegroups.com > (mailto:clojure+unsubscr...@googlegroups.com). > For more options, visit https://groups.google.com/d/optout. -- You received this message because you are subscribed to the Google Groups "Clojure" group. To post to this group, send email to clojure@googlegroups.com Note that posts from new members are moderated - please be patient with your first post. To unsubscribe from this group, send email to clojure+unsubscr...@googlegroups.com For more options, visit this group at http://groups.google.com/group/clojure?hl=en --- You received this message because you are subscribed to the Google Groups "Clojure" group. To unsubscribe from this group and stop receiving emails from it, send an email to clojure+unsubscr...@googlegroups.com. For more options, visit https://groups.google.com/d/optout.
Re: Recommendations for a schema-based data language for use in Hadoop?
FWIW, We use edn (serialized with nippy [1]) in hadoop & it works very well for us: https://github.com/Netflix/PigPen In some places we use maps for the expressiveness and in some we use vectors for more performance. Whatever I lose in raw performance I can trivially throw a few more boxes at, so it makes it a non-issue for us. The flexibility of edn outweighs any performance gains of converting back & forth to another format and having to worry about translation errors. -Matt [1] https://github.com/ptaoussanis/nippy On Tuesday, August 4, 2015 at 7:05 PM, Ryan Schmitt wrote: > Hi Clojure people, > > I'm currently working on some problems in the big data space, and I'm more or > less starting from scratch with the Hadoop ecosystem. I was looking at ways > to work with data in Hadoop, and I realized that (because of how InputFormat > splitting works) this is a use case where it's actually pretty important to > use a data language with an external schema. This probably means ruling out > Edn (for performance and space efficiency reasons) and Fressian (managing the > Fressian caching domain seems like it could get complicated), which are my > default solutions for everything, so now I'm back to the drawing board. I'd > rather not use something braindead like JSON or CSV. > > It seems like there are a few language-agnostic data languages that are > popular in this space, such as: > > * Thrift > * Protobuf > * Avro > > But since the Clojure community has very high standards for data languages, > as well as a number of different libraries that run code on Hadoop, I was > wondering if anyone could provide a recommendation for a fast, extensible, > and well-designed data language to use. (Recommendations of what to avoid are > also welcome.) > -- > You received this message because you are subscribed to the Google > Groups "Clojure" group. > To post to this group, send email to clojure@googlegroups.com > (mailto:clojure@googlegroups.com) > Note that posts from new members are moderated - please be patient with your > first post. > To unsubscribe from this group, send email to > clojure+unsubscr...@googlegroups.com > (mailto:clojure+unsubscr...@googlegroups.com) > For more options, visit this group at > http://groups.google.com/group/clojure?hl=en > --- > You received this message because you are subscribed to the Google Groups > "Clojure" group. > To unsubscribe from this group and stop receiving emails from it, send an > email to clojure+unsubscr...@googlegroups.com > (mailto:clojure+unsubscr...@googlegroups.com). > For more options, visit https://groups.google.com/d/optout. -- You received this message because you are subscribed to the Google Groups "Clojure" group. To post to this group, send email to clojure@googlegroups.com Note that posts from new members are moderated - please be patient with your first post. To unsubscribe from this group, send email to clojure+unsubscr...@googlegroups.com For more options, visit this group at http://groups.google.com/group/clojure?hl=en --- You received this message because you are subscribed to the Google Groups "Clojure" group. To unsubscribe from this group and stop receiving emails from it, send an email to clojure+unsubscr...@googlegroups.com. For more options, visit https://groups.google.com/d/optout.
[ANN] PigPen 0.3.1
We just released PigPen 0.3.1 with a couple of minor improvements: Update cascading version to 2.7.0 Update nippy (for serialization) to 2.10.0 & tune performance PigPen is map-reduce for Clojure, or distributed Clojure. You write idiomatic Clojure code, we run it on thousands of machines using Apache Pig or Cascading. Learn more here: https://github.com/Netflix/PigPen -Matt -- You received this message because you are subscribed to the Google Groups "Clojure" group. To post to this group, send email to clojure@googlegroups.com Note that posts from new members are moderated - please be patient with your first post. To unsubscribe from this group, send email to clojure+unsubscr...@googlegroups.com For more options, visit this group at http://groups.google.com/group/clojure?hl=en --- You received this message because you are subscribed to the Google Groups "Clojure" group. To unsubscribe from this group and stop receiving emails from it, send an email to clojure+unsubscr...@googlegroups.com. For more options, visit https://groups.google.com/d/optout.
Re: [ANN] 2015 State of Clojure Community survey
How about adding a state of Datomic survey? :) -Matt On Sunday, December 6, 2015 at 4:12 PM, Mars0i wrote: > > > On Sunday, December 6, 2015 at 3:26:55 PM UTC-6, Lee wrote: > > > > On Dec 6, 2015, at 3:00 PM, Alex Miller > (javascript:)> wrote: > > > > > Almost all of the questions are optional - if they don't apply to your > > > scenario, you should skip the question. > > > > > > FWIW if I recall correctly (it won't let me back in to check the questions > > now that I finished it) I think that some questions sort of apply but use > > terminology that implies an industrial context (like "in production"). Not > > a big deal but a bit confusing, and maybe if we want to encourage use in > > education and research these could be tweaked. > > I agree with Lee. It's not that there are missing questions. It's that in > some cases I have an answer to the question, but it's not one of the options; > all of the options presume that we're doing certain sorts of things, which > are common in business. Someone doing data manipulation and analysis for > scientific research may be doing a lot of coding that doesn't leave one or > two machines. Likewise for scientific simulations. These uses may not be > part an ongoing process--they're not used to run a business, or a website, or > anything like that. You write something, do a lot with it, then move on to > something else. Or come back and modify it for a related research project. > Clojure's a great language for that sort of thing. > > I'd be happy to go through the survey and suggest a few things to add or > change if that would be helpful. Lee's suggested additional questions seem > worthwhile, too. I agree that some thought would have to be put into > formulating them. > > > > > > > What question would be useful to add re use in academia? > > > > I'd personally be interested in knowing how many people are using Clojure > > for computer science courses, and which ones. Also, how many people are > > using Clojure for research, and in what fields. Also, what tools (e.g. > > IDEs, books, websites) are people using, and what do they think is missing, > > for research and education purposes. > > > > I'm not sure how best to phrase survey questions for these issues. > > > > -Lee > > -- > You received this message because you are subscribed to the Google > Groups "Clojure" group. > To post to this group, send email to clojure@googlegroups.com > (mailto:clojure@googlegroups.com) > Note that posts from new members are moderated - please be patient with your > first post. > To unsubscribe from this group, send email to > clojure+unsubscr...@googlegroups.com > (mailto:clojure+unsubscr...@googlegroups.com) > For more options, visit this group at > http://groups.google.com/group/clojure?hl=en > --- > You received this message because you are subscribed to the Google Groups > "Clojure" group. > To unsubscribe from this group and stop receiving emails from it, send an > email to clojure+unsubscr...@googlegroups.com > (mailto:clojure+unsubscr...@googlegroups.com). > For more options, visit https://groups.google.com/d/optout. -- You received this message because you are subscribed to the Google Groups "Clojure" group. To post to this group, send email to clojure@googlegroups.com Note that posts from new members are moderated - please be patient with your first post. To unsubscribe from this group, send email to clojure+unsubscr...@googlegroups.com For more options, visit this group at http://groups.google.com/group/clojure?hl=en --- You received this message because you are subscribed to the Google Groups "Clojure" group. To unsubscribe from this group and stop receiving emails from it, send an email to clojure+unsubscr...@googlegroups.com. For more options, visit https://groups.google.com/d/optout.
How to disable (or limit) test.check shrinking
I'm using test.check to test a live service. Occasionally it gets a 503 from the service and spends hours trying to shrink the input & reproduce the error. Is there a way to limit the shrinking process to n iterations? Or disable it entirely for some tests? Is there a better approach for using test.check against a live service where transient issues are expected? Should I just retry the actual service call many times before allowing test.check to see it? Thanks, Matt -- You received this message because you are subscribed to the Google Groups "Clojure" group. To post to this group, send email to clojure@googlegroups.com Note that posts from new members are moderated - please be patient with your first post. To unsubscribe from this group, send email to clojure+unsubscr...@googlegroups.com For more options, visit this group at http://groups.google.com/group/clojure?hl=en --- You received this message because you are subscribed to the Google Groups "Clojure" group. To unsubscribe from this group and stop receiving emails from it, send an email to clojure+unsubscr...@googlegroups.com. For more options, visit https://groups.google.com/d/optout.
Re: How to disable (or limit) test.check shrinking
I considered that, but that only partially fixes the issue. If it does actually find a real problem, it’ll never complete because the shrinking takes too long. In the end I’d rather have something not fully shrunk than something that runs forever. -Matt On February 8, 2017 at 1:19:24 PM, Daniel Compton ( daniel.compton.li...@gmail.com) wrote: If the 503 is only returned by failures not relating to what you are testing (e.g. load), then one option might be to catch the exception and retry that request? On Thu, Feb 9, 2017 at 6:48 AM 'Matt Bossenbroek' via Clojure < clojure@googlegroups.com> wrote: > I'm using test.check to test a live service. Occasionally it gets a 503 > from the service and spends hours trying to shrink the input & reproduce > the error. > > Is there a way to limit the shrinking process to n iterations? Or disable > it entirely for some tests? > > Is there a better approach for using test.check against a live service > where transient issues are expected? Should I just retry the actual service > call many times before allowing test.check to see it? > > Thanks, > Matt > > -- > You received this message because you are subscribed to the Google > Groups "Clojure" group. > To post to this group, send email to clojure@googlegroups.com > Note that posts from new members are moderated - please be patient with > your first post. > To unsubscribe from this group, send email to > clojure+unsubscr...@googlegroups.com > For more options, visit this group at > http://groups.google.com/group/clojure?hl=en > --- > You received this message because you are subscribed to the Google Groups > "Clojure" group. > To unsubscribe from this group and stop receiving emails from it, send an > email to clojure+unsubscr...@googlegroups.com. > For more options, visit https://groups.google.com/d/optout. > -- Daniel -- You received this message because you are subscribed to the Google Groups "Clojure" group. To post to this group, send email to clojure@googlegroups.com Note that posts from new members are moderated - please be patient with your first post. To unsubscribe from this group, send email to clojure+unsubscr...@googlegroups.com For more options, visit this group at http://groups.google.com/group/clojure?hl=en --- You received this message because you are subscribed to the Google Groups "Clojure" group. To unsubscribe from this group and stop receiving emails from it, send an email to clojure+unsubscr...@googlegroups.com. For more options, visit https://groups.google.com/d/optout. -- You received this message because you are subscribed to the Google Groups "Clojure" group. To post to this group, send email to clojure@googlegroups.com Note that posts from new members are moderated - please be patient with your first post. To unsubscribe from this group, send email to clojure+unsubscr...@googlegroups.com For more options, visit this group at http://groups.google.com/group/clojure?hl=en --- You received this message because you are subscribed to the Google Groups "Clojure" group. To unsubscribe from this group and stop receiving emails from it, send an email to clojure+unsubscr...@googlegroups.com. For more options, visit https://groups.google.com/d/optout.
Re: How to disable (or limit) test.check shrinking
Perfect - that’s exactly what I’m looking for. Thanks! -Matt On February 9, 2017 at 4:04:17 PM, Gary Fredericks (fredericksg...@gmail.com) wrote: Wrapping the generator in `gen/no-shrink` will give you a generator that pretends it doesn't know how to shrink. On Thursday, February 9, 2017 at 11:55:38 AM UTC-6, Matt Bossenbroek wrote: > > I considered that, but that only partially fixes the issue. If it does > actually find a real problem, it’ll never complete because the shrinking > takes too long. > > In the end I’d rather have something not fully shrunk than something that > runs forever. > > -Matt > > > On February 8, 2017 at 1:19:24 PM, Daniel Compton (daniel.com...@gmail.com) > wrote: > > If the 503 is only returned by failures not relating to what you are > testing (e.g. load), then one option might be to catch the exception and > retry that request? > > On Thu, Feb 9, 2017 at 6:48 AM 'Matt Bossenbroek' via Clojure < > clo...@googlegroups.com> wrote: > >> I'm using test.check to test a live service. Occasionally it gets a 503 >> from the service and spends hours trying to shrink the input & reproduce >> the error. >> >> Is there a way to limit the shrinking process to n iterations? Or disable >> it entirely for some tests? >> >> Is there a better approach for using test.check against a live service >> where transient issues are expected? Should I just retry the actual service >> call many times before allowing test.check to see it? >> >> Thanks, >> Matt >> >> -- >> You received this message because you are subscribed to the Google >> Groups "Clojure" group. >> To post to this group, send email to clo...@googlegroups.com >> Note that posts from new members are moderated - please be patient with >> your first post. >> To unsubscribe from this group, send email to >> clojure+u...@googlegroups.com >> For more options, visit this group at >> http://groups.google.com/group/clojure?hl=en >> --- >> You received this message because you are subscribed to the Google Groups >> "Clojure" group. >> To unsubscribe from this group and stop receiving emails from it, send an >> email to clojure+u...@googlegroups.com. >> For more options, visit https://groups.google.com/d/optout. >> > -- > > Daniel > -- > You received this message because you are subscribed to the Google > Groups "Clojure" group. > To post to this group, send email to clo...@googlegroups.com > Note that posts from new members are moderated - please be patient with > your first post. > To unsubscribe from this group, send email to > clojure+u...@googlegroups.com > For more options, visit this group at > http://groups.google.com/group/clojure?hl=en > --- > You received this message because you are subscribed to the Google Groups > "Clojure" group. > To unsubscribe from this group and stop receiving emails from it, send an > email to clojure+u...@googlegroups.com. > For more options, visit https://groups.google.com/d/optout. > > -- You received this message because you are subscribed to the Google Groups "Clojure" group. To post to this group, send email to clojure@googlegroups.com Note that posts from new members are moderated - please be patient with your first post. To unsubscribe from this group, send email to clojure+unsubscr...@googlegroups.com For more options, visit this group at http://groups.google.com/group/clojure?hl=en --- You received this message because you are subscribed to the Google Groups "Clojure" group. To unsubscribe from this group and stop receiving emails from it, send an email to clojure+unsubscr...@googlegroups.com. For more options, visit https://groups.google.com/d/optout. -- You received this message because you are subscribed to the Google Groups "Clojure" group. To post to this group, send email to clojure@googlegroups.com Note that posts from new members are moderated - please be patient with your first post. To unsubscribe from this group, send email to clojure+unsubscr...@googlegroups.com For more options, visit this group at http://groups.google.com/group/clojure?hl=en --- You received this message because you are subscribed to the Google Groups "Clojure" group. To unsubscribe from this group and stop receiving emails from it, send an email to clojure+unsubscr...@googlegroups.com. For more options, visit https://groups.google.com/d/optout.
