Re: [go-nuts] Re: Two Related Questions. (1) Map Concurrency (2) Making Multidimensional Arrays

[Justin Israel]

On Tue, Sep 13, 2016 at 3:31 PM  wrote:

> I've spent many, many, many hours on this (measured in days, at this
> point) and I'm convinced that Golang is missing a fundamental programming
> capability, that of multidimensional associative arrays. Searching through
> this forum's archives and other sites, I've found that I'm not the first
> one to face this problem and no one who has posted about this has yet
> received a decent response (at least, not one that I have found)! The best
> responses consist either of maps that are, at most, two levels deep or maps
> whose depth is known in advance and declared using something like
> make(map[string]map[string]map[string]map[string]map[string]string) and
> even in those cases, one cannot replace a map with just a string on any of
> those levels.
>
> This is a MAJOR missing feature in Golang.
>
> In PHP, for example, multidimensional associative arrays are declarable
> and usable in a single line of code. In Javascript, Objects can serve that
> purpose and those serve as the foundation of pretty much the entire
> language. HTML is built on the tree structure, which is essentially just a
> multimentional array. The Internet is essentiallly built on it
> multidimensional associative arrays! Take XML or JSON, both of which are
> essentially multidimensional associative arrays. Take the file/folder
> system on your desktop computers and on servers as another example. Take
> relational database recordsets. I mean, the list goes on and on and on! How
> can Golang not have a simple way to initialize, manipulate and maintain
> multidimensional associative arrays?
>
> This is such a big problem that I've even spent time looking into
> alternate options to Golang, such as Clojure, Erlang and Elixir, and I'm
> quite certain others have followed my route. I am not about to give up on
> Golang just yet, because it would mean a tremendous amount of rewriting
> what I've already done and learning yet another new language, but this is a
> very serious missing piece in Golang.
>
> It's incredible that this has gone overlooked in Golang for so long. The
> posts I found were from as early as 2012! This is a very disappointing
> discovery. I have found many things about Golang that I really like and
> many things about Golang that I really don't like, but the lack of
> multidimensional associative arrays beats out even the lack of decent
> global error handling! By the way, while I'm at it are there any libraries
> for that?
>
> Is there any of doing multidimensional associative arrays in Golang
> (beyond two levels and without specifying the depth of the array or map in
> advance) or am I wasting my time on this problem? If not, does anyone know
> if this feature is planned for an upcoming version of Golang?
>

When you say this is a missing feature of Go and that you can't imagine how
Go programmers have expressed solutions to common problems without it, I
interpret that as (and I apologize that you probably won't agree) "this
approach that I do in PHP or Javascript does work exactly the same way in
Go, so I am calling it a missing feature". The thing is, PHP and Javascript
are both dynamically typed languages. They would both allow you to shove
any types that you want, mixed, into data structures. Thus you would be
able to express your map of strings to either a string or another map. In a
statically-type language like Go, you have to either be explicit about the
concrete types you are storing, or you have to use interfaces/reflection in
order to mix types with common behaviour and then inspect and extract the
concrete type or call the interface methods.

Because I am more familiar with Python than PHP/JS, let me use that as a
comparison. In Python you can create a dictionary with any kind of hashable
key type and any value, mixed:

aDict = {
(1, 2, 'c'): "value",
'string': {'another': 'value'},
1: ['a', 'b', 3],
}

​
But really every object in Python is just a C wrapper object called a
PyObject (that is, in CPython). Python will then dynamically evaluate the
type and resolve methods and operations on them as needed. When you have a
mixed dictionary (multi-dimensional associative array) of more nested dicts
and string values, it allows you to do things like:   aDict['a']['b']['c']
because on-the-fly, Python would try and treat each value as a dict and it
would either work at runtime. or raise an exception if the object stored at
that point did not support the operation.
Go, being a statically typed language, can have the mixed types, but the
1-to-1 equivalent (or closest translation) of a "GoObject" (PyObject) would
be interfaces. And then you need to manually assert the type, whereas
Python would be doing it dynamically when you simply try to use the object,
because that is how it does it for all objects whenever you access them.

But honestly, with all the examples you listed in the world of
tree-structured data, that does not mean it 

Re: [go-nuts] Re: Two Related Questions. (1) Map Concurrency (2) Making Multidimensional Arrays

[Dan Kortschak]

Another way that you can use (based on a recursive map[string]T - where
T is an interface type - can be seen here where I use it create a
filesystem of arbitrary behaviours).

https://github.com/ev3go/sisyphus

The general case though uses map[string]interface{}. Each time you want
to add a layer at "basepath", you navigate the tree and add a newly made
map[string]interface{} at that key. See
https://github.com/ev3go/sisyphus/blob/master/fs.go for code that does
this kind of thing.

-- 
You received this message because you are subscribed to the Google Groups 
"golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to golang-nuts+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: [go-nuts] Re: Two Related Questions. (1) Map Concurrency (2) Making Multidimensional Arrays

[Dan Kortschak]

On Mon, 2016-09-12 at 20:26 -0700, davidmiller...@gmail.com wrote:
> Well, PHP is also free. It can do multidimensional associative arrays
> and much more. What good is Golang's tremendous efficiency if it isn't
> capable of doing what is needed in the first place? 

Depends on how much work you want to put in.

> Thank you for your response.
> 
Brad Fitzpatrick sent an arbitrarily deep map approach some time back
that might do what you want without any real additional programming.

A map[interface{}]string will do what you want if you make sure the keys
are always equalable. The approach Brad used went something like this.

https://play.golang.org/p/TTxd6tTrQI


-- 
You received this message because you are subscribed to the Google Groups 
"golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to golang-nuts+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: [go-nuts] Re: Two Related Questions. (1) Map Concurrency (2) Making Multidimensional Arrays

[davidmiller124]

Thank you for clarifying. My response is inline.

On Monday, September 12, 2016 at 11:20:10 PM UTC-4, kortschak wrote:
>
> On Mon, 2016-09-12 at 20:06 -0700, davidmi...@gmail.com  
> wrote: 
> > If I understand that correctly, my index key is now "Folder_1 
> > Folder_2 
> > Folder_3 Folder_4" and my value is now "File_Name_1". How would I now 
> > assign "File_Name_2" to Folder_2, for example? 
>
> The key would be []string{"Folder_1", "Folder_2", "Folder_3", 
> "Folder_4"} 
>
>
That's what I thought. That's not what is needed.
 

> > In other words, how is this any different than using ":" or "/" to 
> > fake the behavior of a multidimensional array? 
>
> It's not except that you can use ":" and "/" in the names. 
>
> > In this case, I'd be simply using spaces, which would technically mean 
> > that I couldn't have multi word folders with spaces in their names 
> > because the spaces would be considered a level divider. 
>
> If it's a multidimensional array where the number of dimensions varies, 
> then this is probably not what you want, 


You are correct. It's not what I want. The number of dimensions does vary 
and is unknown at the outset when the map is declared.

though you could use "" as a 
> null key entry. So to answer the question about "File_Name_2" in g 
> "Folder_2" 
>
> m[[4]string{"Folder_1", "Folder_2", "", ""}] = "File_Name_2" 
>
> It doesn't seem very satisfying, but you get it essentially for free. In 
> many cases, you get what you pay for. 
>
>
Well, PHP is also free. It can do multidimensional associative arrays and 
much more. What good is Golang's tremendous efficiency if it isn't capable 
of doing what is needed in the first place? 

Thank you for your response.

-- 
You received this message because you are subscribed to the Google Groups 
"golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to golang-nuts+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: [go-nuts] Re: Two Related Questions. (1) Map Concurrency (2) Making Multidimensional Arrays

[Dan Kortschak]

On Mon, 2016-09-12 at 20:06 -0700, davidmiller...@gmail.com wrote:
> If I understand that correctly, my index key is now "Folder_1
> Folder_2 
> Folder_3 Folder_4" and my value is now "File_Name_1". How would I now 
> assign "File_Name_2" to Folder_2, for example?

The key would be []string{"Folder_1", "Folder_2", "Folder_3",
"Folder_4"}

> In other words, how is this any different than using ":" or "/" to
> fake the behavior of a multidimensional array?

It's not except that you can use ":" and "/" in the names.

> In this case, I'd be simply using spaces, which would technically mean
> that I couldn't have multi word folders with spaces in their names
> because the spaces would be considered a level divider.

If it's a multidimensional array where the number of dimensions varies,
then this is probably not what you want, though you could use "" as a
null key entry. So to answer the question about "File_Name_2" in g
"Folder_2"

m[[4]string{"Folder_1", "Folder_2", "", ""}] = "File_Name_2"

It doesn't seem very satisfying, but you get it essentially for free. In
many cases, you get what you pay for.

-- 
You received this message because you are subscribed to the Google Groups 
"golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to golang-nuts+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: [go-nuts] Re: Two Related Questions. (1) Map Concurrency (2) Making Multidimensional Arrays

[davidmiller124]

Thank you for clarifying that, but how does that resolve my problem?

The output I get is.

map[[Folder_1 Folder_2 Folder_3 Folder_4]:File_Name_1]

If I understand that correctly, my index key is now "Folder_1 Folder_2 
Folder_3 Folder_4" and my value is now "File_Name_1". How would I now 
assign "File_Name_2" to Folder_2, for example?

In other words, how is this any different than using ":" or "/" to fake the 
behavior of a multidimensional array? In this case, I'd be simply using 
spaces, which would technically mean that I couldn't have multi word 
folders with spaces in their names because the spaces would be considered a 
level divider.

On Monday, September 12, 2016 at 10:54:36 PM UTC-4, kortschak wrote:
>
> https://play.golang.org/p/6ZyybAKWpp 
>
>
> On Mon, 2016-09-12 at 18:50 -0700, davidmi...@gmail.com  
> wrote: 
> > I tried that. I can't seem to get it to work. I'm clearly doing 
> > something 
> > wrong, probably because I am misunderstanding Tamás's response. 
> > 
> > Test_Map := make(map[[4]string]string) 
> > 
> > I've tried this way. 
> > 
> > 
> > Test_Map["Folder_1"]["Folder_2"]["Folder_3"]["Folder_4"] = 
> > "File_Name_1" 
>
>
>

-- 
You received this message because you are subscribed to the Google Groups 
"golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to golang-nuts+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: [go-nuts] Re: Two Related Questions. (1) Map Concurrency (2) Making Multidimensional Arrays

[Dan Kortschak]

https://play.golang.org/p/6ZyybAKWpp


On Mon, 2016-09-12 at 18:50 -0700, davidmiller...@gmail.com wrote:
> I tried that. I can't seem to get it to work. I'm clearly doing
> something 
> wrong, probably because I am misunderstanding Tamás's response.
> 
> Test_Map := make(map[[4]string]string)
> 
> I've tried this way.
> 
> 
> Test_Map["Folder_1"]["Folder_2"]["Folder_3"]["Folder_4"] =
> "File_Name_1"


-- 
You received this message because you are subscribed to the Google Groups 
"golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to golang-nuts+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: [go-nuts] Re: Two Related Questions. (1) Map Concurrency (2) Making Multidimensional Arrays

[Justin Israel]

On Tue, Sep 13, 2016 at 7:42 AM  wrote:

>
> *One of the primary benefits of concurrency is that if one procedure gets
> stuck or slows down, other procedures can keep moving along, evading the
> bottleneck. Once you throw a shared bottleneck into the concurrent
> routines, one loses out on the benefit of using concurrent procedures.*
>
> *It's like if cars were driving down the highway in two lanes and the
> lanes merged for just a moment and then separated again into two lanes. If
> there were no exits before the merge (between the cars approaching the
> merge and the merge), then using two lanes would be pointless, since they
> all have to merge with each other anyway at that bottleneck. When they get
> to the merge point and have to wait their turn. It doesn't matter that they
> were concurrent (even parallel) before the bottleneck. They might as well
> have not as been concurrent in the first place! In that case, there would
> be no merge and everyone could just travel in a straight line at a steady
> speed! No bottleneck! Notice how there can be two traffic points at a toll
> on a toll road. The first is when someone takes a long time to pay. The
> second is the merge after the toll. If the first can be eliminated,
> splitting the lanes up into multiple toll lanes and then merging them again
> would only cause traffic and not prevent it. In my scenario, a Golang map
> would apparently work, not like a toll, but like the merge after the toll.*
>
> *It doesn't really matter how long the merge takes or how short a time one
> locks the map. If it's so fast that it can be done in "single threaded"
> mode (concurrently speaking) anyway, why use concurrency? The fact that no
> other procedure can use the map while it is locked potentially forces a
> full stop on all other procedures waiting to use the map. As just stated,
> it's as if all such procedures have gone back to "single threaded" mode
> (concurrently speaking).*
>

This and your previous impressions of concurrency seem like some pretty
wild ideas, making me feel like you may not have a complete grasp of
concurrency vs parallelism. A map doesn't magically become "concurrency
safe". It requires either a form of locking, or to be a lock-free
implementation. Go maps are not concurrent by design. This makes them
faster for when you aren't using them concurrently. And when you are, you
are required to synchronise access to them. Now, I don't have much PHP
experience, but if I understand correctly, PHP doesn't put you in a
situation where a global associative array can even be accessible across
requests because it is "shared nothing". So you are finding yourself being
introduced to one of a number of languages that expose you to real
concurrency concerns with shared data structures.

I also feel the example of the cars on the highway doesn't properly
describe concurrency. It implies that the cars have absolutely nothing to
do, aside from driving in a straight line next to each other. But really
they are free to do all kinds of other work before and after the point that
the need to merge (synchronize) for a moment. Your requests can do all
kinds of things in their own goroutines up to the point where they need to
read or write a shared map. For that extremely brief moment, the access to
the map is protected. That does not stop other requests from continuing to
do the other work they intended to do. It only blocks the access to the
map. Now lets assume that the map implementation was already safe for
concurrent access... that could STILL mean the access would block. Only
that you aren't the one having to arrange it.

Point being, your take on concurrency implies that if any kind of
synchronization occurs in the system, that the entire concept of
concurrency is broken. It is just not true. What you aim to do is have as
much work as possible happen in parallel. Sometimes it cannot, either
because there are not enough physical cores, or there are synchronization
points in the code between various tasks that want to happen.

See: Rob Pike - 'Concurrency Is Not Parallelism'




>
>>
>>
>>
>>> *2) Also, how would one build a multidimensional array with a string
>>> index (or even a map of maps) for something like a large file directory
>>> where the number of files, folders and levels are unknown at the outset
>>> (without repeatedly using the append function that apparently uses memory
>>> on an exponential scale of 2^x)? I've figured out how to do
>>> map[string]map[string]map[string]string, but one would have to know how
>>> many levels there would be, in advance, to be able to use something like
>>> that. Also, when I've tried that, I can't put a string on any level except
>>> for the last one, since the only data type that any level other than the
>>> last one will accept is another map! What if the map turns out to be only
>>> three levels deep for one path and five levels deep for