Hi Patrick,
Sorry for requesting code you already posted.
I'm quite unsure about what functions (in particular get-key-instance)
should do.
Btw there's two easy ways to test if a collection is empty in clojure
(rather than (= 0 (count coll))):
(empty? coll) ;; true when empty
or (seq coll) ;; true when not empty
To understand your code, I tried to rewrite some functions in more
idiomatic code:
(defn process-row-data-file [inp-key-names-list]
(with-open [rdr (reader my-row-data-file)]
(let [inp-row-data (map process-row-str (line-seq rdr))]
(reduce #(update-map-list %1 (produce-date-rn-array %2)
%2 inp-key-names-list)
{} inp-row-data))))
;; identical to yours
(defn process-row-str [inp-row-str]
(let [[row-nb & inner-array] (.split inp-row-str "@")]
(cons row-nb (map #(.split % ",") inner-array))))
;; but it would be better to build a map to pair keys and values:
(defn process-row-str [inp-row-str]
(let [[row-nb & inner-array] (.split inp-row-str "@")
[key-list value-list row] (map #(.split % ",") inner-array)]
[row-nb (zipmap key-list value-list)]))
;; (process-row-str "4...@foo,b...@13,48") => ["42" {"bar" "48", "foo" "13"}]
(defn do-the-date-bigint [inp-date-string]
(let [[dd mm yy] (.split inp-date-string "/")]
(str "20" yy mm dd)))
(defn produce-date-rn-array [[row data]]
(if-let [date (data "Date")]
[(do-the-date-bigint date) row]
[]))
I roughly understand what update-map-list (you should rewrite it using
reduce) does but I'm lost when it comes to get-key-instance: what do you
pass to it and what does it return?
Once it will be clearly undestood what each function does it will be
easier to spot your problem.
hth,
Christophe
pjfitz a écrit :
> Hi,
> Here is the code. Each row of data is a string with @ between the
> three types of data i mentioned.
> -Row Number
> -Heading info List
> -Actual Info List
> The Heading Info List is created by splitting on , as is the Actual
> info list.
> tks,
> PJ
>
>
> (import '(java.io InputStreamReader OutputStreamWriter))
> (use '[clojure.contrib.duck-streams])
> (import '(java.io File))
>
> (def my-row-data-file "e:/SoccerData/rowdata.txt")
> (def my-map-file "e:/SoccerData/map-data.txt")
>
> (defn get-idx-of [inp-data inp-name]
> (loop [my-count 0 inner-data inp-data]
> (cond
> (= (count inner-data) 0) -1
> (= inp-name (first inner-data)) my-count
> :else
> (recur (+ 1 my-count) (rest inner-data))
> )
> )
> )
>
> (defn do-the-date-bigint [inp-date-string]
> (let [date-array (.split inp-date-string "/")]
> (str "20" (nth date-array 2)
> (nth
> date-array 1)
> (nth
> date-array 0)
> )
> )
> )
>
> ;;
>
> (defn get-key-instance [inp-key-names inp-row-data]
> (loop [my-inner-vector (first inp-key-names) inner-key-names (rest
> inp-key-names)]
> (cond
> (= (count inner-key-names) 0) (cons inp-key-names (vec
> (reverse my-
> inner-vector)))
> :else
> (let [index-row (get-idx-of (nth inp-row-data 1) (first
> inner-key-
> names))]
> (if (== index-row -1) ()
> (recur (cons (nth (last inp-row-data) index-row)
>
> my-inner-vector)
> (rest
> inner-key-names)
> )
> )
> )
> )
> )
> )
>
> (defn produce-date-rn-array [inp-row-data]
> (try
> [(do-the-date-bigint (nth (nth inp-row-data 2) (get-idx-of (nth inp-
> row-data 1) "Date")))
> (nth inp-row-data 0)] (catch Exception e []
> )
> )
> )
>
> (defn update-map-list [inp-map inp-new-array inp-row-data inp-key-
> names-list]
> (loop [inner-key-names-list inp-key-names-list inner-map inp-map]
> (cond
> (= (count inner-key-names-list) 0) inner-map
> :else
> (let [inner-key-instance (get-key-instance (first
> inner-key-names-
> list) inp-row-data)]
> (recur
> (rest inner-key-names-list)
> (assoc-in inner-map inner-key-instance
> (sort-by #(first %) (cons inp-new-array
> (get-in inner-map inner-
> key-instance)))
> )
> )
> )
> )
> )
> )
>
> (defn process-row-str[inp-row-str]
> (loop [inner-array (rest (.split inp-row-str "@")) inner-list [(first
> (.split inp-row-str "@"))]]
> (cond
> (= (count inner-array) 0) (reverse inner-list)
> :else
> (recur (rest inner-array)
> (cons (.split (first
> inner-array) ",") inner-list)
> )
> )
> )
> )
>
> (defn process-row-data-file [inp-key-names-list]
> (with-open [rdr (reader my-row-data-file)]
> (loop [inner-row-data (line-seq rdr) my-inner-map {}]
> (cond
> (= (count inner-row-data) 0) my-inner-map
> :else
> (recur (rest inner-row-data)
>
> (update-map-list my-inner-map
>
> (produce-date-rn-array
> (process-row-str (first inner-
> row-data)))
>
> (process-row-str (first
> inner-row-data))
>
> inp-key-names-list
> )
> )
> )
> )
> )
> )
>
>
>
>
>
> On Apr 27, 9:34 pm, "Dimiter \"malkia\" Stanev" <mal...@gmail.com>
> wrote:
>
>> Unless you provide some kind of isolated source code that reproduces
>> the problem, I don't think it is going to be easy to help you out.
>>
>> But try adding "(doall ..)" to some your "map" calls and others.
>>
>> Even better, look at this excellent explanation why this might be
>> happening by Cristophe
>> Grand:http://groups.google.com/group/clojure/browse_frm/thread/43f9a716f1f9...
>>
>> Also check whether you don't have recursive functions (replace them
>> with loop/recur), or mutually recursive ones (for them, you would have
>> to use mutual exclusion). But I doubt that was the problem, more
>> likely the default lazy mode of clojure, and what Cristophe was
>> talking about is probably connected to your problem.
>>
>> On Apr 27, 1:07 pm, "fitzpatrick...@googlemail.com"
>>
>> <fitzpatrick...@googlemail.com> wrote:
>>
>>> Hi,
>>> I am working with a very long list and populating it to a map. What i
>>> have is three pieces of data from the list;
>>> -Row Number
>>> -Heading info List
>>> -Actual Info List
>>> note: both the Heading Info List and Actual Info List have the same
>>> number of elements as each heading name will have a corresponding
>>> actual value.
>>>
>>> I define a headingnamelist which is a list of headings to be used to
>>> generate the keys.
>>>
>>> I iterate through the list and for each row look up each of the
>>> heading names in the Heading Info List to get an index of where a
>>> particular element should be in the Actual Info List and then i go and
>>> get this from the Actual Info List and use this as part of my key.
>>> With the key i write the Row Number to the map. If the key is already
>>> there i just cons it to the exiting map value.
>>>
>>> As i said it is quite a large list and i keep getting a stack
>>> overflow. For comparison purposes i ran a similar program in python
>>> and it ran in a matter of seconds.
>>>
>>> Has anybody any ideas why this is happening?
>>>
>>> tks,
>>> PJ
>>>
> >
>
>
--
Professional: http://cgrand.net/ (fr)
On Clojure: http://clj-me.blogspot.com/ (en)
--~--~---------~--~----~------------~-------~--~----~
You received this message because you are subscribed to the Google Groups
"Clojure" group.
To post to this group, send email to clojure@googlegroups.com
To unsubscribe from this group, send email to
clojure+unsubscr...@googlegroups.com
For more options, visit this group at
http://groups.google.com/group/clojure?hl=en
-~----------~----~----~----~------~----~------~--~---
