*I already made your point in my previous post.....* *"That's not to say that there aren't other potential bottlenecks being evaded by using concurrent routines that do no have the shared map problem....."*
*I think you made have taken my comment that "**Once you throw a shared bottleneck into the concurrent routines, one loses out on the benefit of using concurrent procedures." further than I intended. I am aware of the difference between parallel and concurrent and have already watched the video you referred me to a while ago. The difference between the two really isn't that hard to understand and I was actually surprised that there was a whole 30 minute video on it.* *The problem I'm having is that the map manipulation would be among the core functionality of the concurrent procedures. Essentially, the cars are on a very short road and do (almost) nothing except go through a merge.* *The answer I'm seeking now is whether or not I can avoid the locking problem by declaring the map within the goroutine so that multiple, independent, instances will be declared. I'm wondering if those would be safe within the context (and only within the context) of each of those goroutines or whether golang will be using the same shared space for each instance, causing problematic behavior.* On Monday, September 12, 2016 at 7:47:20 PM UTC-4, Justin Israel wrote: > > > On Tue, Sep 13, 2016 at 7:42 AM <davidmi...@gmail.com <javascript:>> > wrote: > >> >> *One of the primary benefits of concurrency is that if one procedure gets >> stuck or slows down, other procedures can keep moving along, evading the >> bottleneck. Once you throw a shared bottleneck into the concurrent >> routines, one loses out on the benefit of using concurrent procedures.* >> >> *It's like if cars were driving down the highway in two lanes and the >> lanes merged for just a moment and then separated again into two lanes. If >> there were no exits before the merge (between the cars approaching the >> merge and the merge), then using two lanes would be pointless, since they >> all have to merge with each other anyway at that bottleneck. When they get >> to the merge point and have to wait their turn. It doesn't matter that they >> were concurrent (even parallel) before the bottleneck. They might as well >> have not as been concurrent in the first place! In that case, there would >> be no merge and everyone could just travel in a straight line at a steady >> speed! No bottleneck! Notice how there can be two traffic points at a toll >> on a toll road. The first is when someone takes a long time to pay. The >> second is the merge after the toll. If the first can be eliminated, >> splitting the lanes up into multiple toll lanes and then merging them again >> would only cause traffic and not prevent it. In my scenario, a Golang map >> would apparently work, not like a toll, but like the merge after the toll.* >> >> *It doesn't really matter how long the merge takes or how short a time >> one locks the map. If it's so fast that it can be done in "single threaded" >> mode (concurrently speaking) anyway, why use concurrency? The fact that no >> other procedure can use the map while it is locked potentially forces a >> full stop on all other procedures waiting to use the map. As just stated, >> it's as if all such procedures have gone back to "single threaded" mode >> (concurrently speaking).* >> > > This and your previous impressions of concurrency seem like some pretty > wild ideas, making me feel like you may not have a complete grasp of > concurrency vs parallelism. A map doesn't magically become "concurrency > safe". It requires either a form of locking, or to be a lock-free > implementation. Go maps are not concurrent by design. This makes them > faster for when you aren't using them concurrently. And when you are, you > are required to synchronise access to them. Now, I don't have much PHP > experience, but if I understand correctly, PHP doesn't put you in a > situation where a global associative array can even be accessible across > requests because it is "shared nothing". So you are finding yourself being > introduced to one of a number of languages that expose you to real > concurrency concerns with shared data structures. > > I also feel the example of the cars on the highway doesn't properly > describe concurrency. It implies that the cars have absolutely nothing to > do, aside from driving in a straight line next to each other. But really > they are free to do all kinds of other work before and after the point that > the need to merge (synchronize) for a moment. Your requests can do all > kinds of things in their own goroutines up to the point where they need to > read or write a shared map. For that extremely brief moment, the access to > the map is protected. That does not stop other requests from continuing to > do the other work they intended to do. It only blocks the access to the > map. Now lets assume that the map implementation was already safe for > concurrent access... that could STILL mean the access would block. Only > that you aren't the one having to arrange it. > > Point being, your take on concurrency implies that if any kind of > synchronization occurs in the system, that the entire concept of > concurrency is broken. It is just not true. What you aim to do is have as > much work as possible happen in parallel. Sometimes it cannot, either > because there are not enough physical cores, or there are synchronization > points in the code between various tasks that want to happen. > > See: Rob Pike - 'Concurrency Is Not Parallelism' > <https://www.youtube.com/watch?v=cN_DpYBzKso> > > > >> >>> >>> >>> >>>> *2) Also, how would one build a multidimensional array with a string >>>> index (or even a map of maps) for something like a large file directory >>>> where the number of files, folders and levels are unknown at the outset >>>> (without repeatedly using the append function that apparently uses memory >>>> on an exponential scale of 2^x)? I've figured out how to do >>>> map[string]map[string]map[string]string, but one would have to know how >>>> many levels there would be, in advance, to be able to use something like >>>> that. Also, when I've tried that, I can't put a string on any level except >>>> for the last one, since the only data type that any level other than the >>>> last one will accept is another map! What if the map turns out to be only >>>> three levels deep for one path and five levels deep for another path?** >>>> I've >>>> tried the following, which I found online, but I can't dynamically add any >>>> new folders to the directory with any kind of ease. It seems that >>>> initializing the struct is the only way to get data into the struct >>>> without >>>> declaring a new struct for every manipulation!* >>>> >>>> >>>> *(Below code from >>>> **http://stackoverflow.com/questions/19080116/golang-tree-like-solution-for-filesystem >>>> >>>> <http://stackoverflow.com/questions/19080116/golang-tree-like-solution-for-filesystem>)* >>>> >>>> >>>> package Websocket >>>> >>>> import "fmt" >>>> >>>> type File struct { >>>> Name string >>>> } >>>> >>>> type Folder struct { >>>> Name string >>>> Files []File >>>> Folders []Folder >>>> } >>>> >>>> func main() { >>>> root := Folder{ >>>> Name: "Root", >>>> Files: []File{ >>>> {"One"}, >>>> {"Two"}, >>>> }, >>>> Folders: []Folder{ >>>> { >>>> Name: "Empty", >>>> }, >>>> }, >>>> } >>>> fmt.Printf("Root %#v\n", root) >>>> } >>>> >>>> >>>> *Given the above, how would i now add another folder to the directory >>>> tree. Among many other variations, I've tried the following.....* >>>> >>>> >>>> *root.Folders[0].Name = "MyFirstFolder"* >>>> >>>> >>>> *.....but nothing I have tried has worked.* >>>> >>>> >>>> *What I've been trying to do is set up an array that looks like the >>>> following.....* >>>> >>>> >>>> *MyArray["Number of Sub Items"] = 6* >>>> >>>> *MyArray["Type of Sub Items"] = "Folders"* >>>> >>>> *MyArray[0] = "Folder 1"* >>>> >>>> *MyArray[1]** = "Folder 2"* >>>> >>>> *MyArray[2]** = "Folder 3"* >>>> >>>> *MyArray[3]** = "Folder 4"* >>>> >>>> *MyArray[4]** = "Folder 5"* >>>> >>>> *MyArray[6]** = "Folder 6"* >>>> >>>> >>>> *MyArray[3]**["Number of Sub Items"]** = 3* >>>> >>>> *MyArray["Type of Sub Items"] = "Folders"* >>>> >>>> *MyArray[3][0] = "Folder 1"* >>>> >>>> *MyArray[3][1]** = "Folder 2"* >>>> >>>> *MyArray[3][2]** = "Folder 3"* >>>> >>>> >>>> *Note that this problem is not exclusive to directory structure. Using >>>> directory structure is just a simple way to describe the problem I'm >>>> having.* >>>> >>>> >>>> *Alternatively**, are there any libraries that can do the above so that I >>>> don't have to "reinvent the wheel"?* >>>> >>>> >>>> >>> For not reinventing the wheel, take a look at http.File, FileSystem, >>> and Dir >>> >> >> *As I said above..... "Note that this problem is not exclusive to >> directory structure. Using directory structure is just a simple way to >> describe the problem I'm having."* >> >> *I used the "File/Folder" description as an easier way to describe the >> scenario. I could have used family trees, for example, as well. Any tree >> structure where a variable could either be a string or another array/map >> would have been sufficient. I could have also referred to Javascript >> Objects as well to describe what I was referring to. I am not concerned, >> specifically, with the system's file structure. * >> >> >>> For using a map: why not use the path (with / as separator) as the key? >>> >> >> *This is not a terrible suggestion. I found a similar suggestion online >> last night (after I posted here) using ":" between levels. However, before >> using "workarounds", I'd like to find out how Golang intended developers to >> accomplish this objective. Multidimensional arrays (and trees) with string >> indexes are pretty fundamental to programming. For example, just about any >> recordset returned from a database will be returned with a field (or >> column) name and value, which would contain a string as the field (or >> column) name. In that case, it would be helpful to access the data in each >> row as part of an associative array, while each row would also be a sub row >> of the larger array containing all of the rows in the recordset. **I >> figured there must be a way to use them properly in Golang without breaking >> concurrency.* >> >> *Before resorting to using something like ":" or "/" for the key, I'd >> like to see if there's a "proper" way to accomplish this in Golang first.* >> >> >>> Or implement a tree with pointers, if you really need it. >>> >> >> *Wouldn't implementing a tree with pointers have the same problem as >> implementing a map? Wouldn't writing to the tree cause other goroutines to >> see the edited version of the tree, rather than their own, exclusive, >> version of the map?* >> >> *Also, how does one implement a tree using strings as indexes if not by >> using a map of maps?* >> >> >>> >>> What are you wishing to accompish? >>> >>> >> *I'd like to accomplish exactly what I've stated. When working with data, >> I end up having to manipulate many types of [key]item lists, many of which >> can have sub lists, such as [key][key]item and where [key] is often a >> string, not an int. This is not a "one case scenario" where an alternate >> approach would be acceptable (like your file system response). It's a >> common scenario that I come across that has to be resolved. * >> >> *I have been exploring the "append" option, but I am concerned about the >> exponential doubling of capacity. Wouldn't it have made more sense to have >> another variable in the function to specify how much should be added on if >> the capacity has been reached? That parameter could have been either an >> int, which would indicate how much capacity to add when the limit is >> reached, or an option, such as "double the existing amount" or "add another >> set of the initial amount". This last question is mostly a rhetorical >> question that only the Golang team of developers could probably answer.* >> >> *Again, thank you for your response. If I've understood your answer to >> the first question, then you have sufficiently answered it. Thank you!* >> >> *As for the second question, I'm still trying to figure out the best way >> to solve that problem.* >> >> -- >> You received this message because you are subscribed to the Google Groups >> "golang-nuts" group. >> To unsubscribe from this group and stop receiving emails from it, send an >> email to golang-nuts...@googlegroups.com <javascript:>. >> For more options, visit https://groups.google.com/d/optout. >> > -- You received this message because you are subscribed to the Google Groups "golang-nuts" group. To unsubscribe from this group and stop receiving emails from it, send an email to golang-nuts+unsubscr...@googlegroups.com. For more options, visit https://groups.google.com/d/optout.
