4.5.0 kernel still gets a NULL ptr. I have kernel core dumps if anyone knows what to look for. I am now trying 4.2 kernel.
-----Original Message----- From: Rune Torgersen [mailto:ru...@innovsys.com] Sent: Wednesday, March 23, 2016 12:48 PM To: 'Erik Hugne' Cc: tipc-discussion@lists.sourceforge.net Subject: Re: [tipc-discussion] tipc nametable update problem Interfaces are renamed, yes, but that should all have been done before TIPC is loaded and configured. So I have been testing different kernels, and the nametable update problems only seem to occur with the Ubuntu 4.4.0 kernel, which is based on the 4.4.5 mainline. Older kernels (3.14.1 and 4.2.0) does not seem to have problems (at least after reboot, no nametable issues, and all entries seem to be on both sides). Also 4.5.0 kernel seem to work. I’ll give it a try and see if it is stable enough (looks like it also might have a fix for the NULL ptr crash I saw). Another interesting thing I saw on one reboot, was that one unit got completely invalid entries from the other. [ 29.916004] Dropping name table update (0) of {0, 0, 0} from <1.1.2> key=3210283144 [ 29.916010] Dropping name table update (0) of {442042504, 4294901760, 0} from <1.1.2> key=0 [ 29.916013] Dropping name table update (0) of {3260614792, 4294955122, 3243837576} from <1.1.2> key=2743205887 [ 29.916016] Dropping name table update (0) of {0, 0, 0} from <1.1.2> key=0 [ 29.916019] Dropping name table update (0) of {0, 0, 0} from <1.1.2> key=2184 [ 29.916022] Dropping name table update (0) of {4294911232, 0, 29952} from <1.1.2> key=57427 [ 29.916025] Dropping name table update (0) of {3831105672, 4294959155, 442042504} from <1.1.2> key=89786504 on 1.1.1: 294907136 0 49389 <1.1.2:3260614792> 3260614792 cluster 256 0 0 <1.1.2:0> 0 cluster 4294911232 0 8418048 <1.1.2:268959744> 268959744 cluster 4294957547 3260614792 4294934691 <1.1.2:1312882922> 1312882922 cluster 4294928877 3260614792 4294901760 <1.1.2:0> 0 cluster 4294904576 0 38 <1.1.2:1062668424> 1062668424 cluster 2816 0 63724 <1.1.2:3260614792> 3260614792 cluster 1097732991 18669 3260614792 <1.1.2:4294949099> 4294949099 cluster while on 1.1.2 those addresses does not exist. From: Erik Hugne [mailto:erik.hu...@gmail.com] Sent: Wednesday, March 23, 2016 12:07 PM To: Rune Torgersen Cc: tipc-discussion@lists.sourceforge.net Subject: Re: [tipc-discussion] tipc nametable update problem When an update is received, bit cannot immediately be applied to the local nametable, we retain it for a few seconds in a backlog queue. Then for each subsequent update received (that may have cleared up the conflict) we try to apply any update stored in the backlog. The timeout can be set with sysctl -w tipc.named_timeout=xxx Default is 2000ms. So clock drift does not matter. I'm guessing that the nametable updates are dropped on the sending side. Are there any interface renaming going on after tipc is enabled? //E On Mar 23, 2016 17:04, "Rune Torgersen" <ru...@innovsys.com<mailto:ru...@innovsys.com>> wrote: How much clock drift between units does the nametable update allow? On one of the test units, the clock was off by about a second between them. -----Original Message----- From: Rune Torgersen [mailto:ru...@innovsys.com<mailto:ru...@innovsys.com>] Sent: Tuesday, March 22, 2016 10:58 AM To: tipc-discussion@lists.sourceforge.net<mailto:tipc-discussion@lists.sourceforge.net> Subject: Re: [tipc-discussion] tipc nametable update problem Still having nametable update problems (Using TIPC_CLUSTER_SCOPE) Here is an except of tipc-config -nt on both systems: address 1.1.1: 104 1025 1025 <1.1.1:3540751351> 3540751351 cluster 104 65537 65537 <1.1.1:4046699456> 4046699456 cluster 104 131073 131073 <1.1.2:59828181> 59828181 cluster 104 16777984 16777984 <1.1.1:3135589675> 3135589675 cluster 104 33555200 33555200 <1.1.2:2193437365> 2193437365 cluster Address 1.1.2: 104 131073 131073 <1.1.2:59828181> 59828181 cluster 104 33555200 33555200 <1.1.2:2193437365> 2193437365 cluster So in this case 1 sees all address 2 has published, while 2 is not seeing the addesses from 1. 2 was rebooted to make this happen. Is tere a possibility I'm calling tipc-config too early, and the interface is not yet up, or is this still the same roblem I saw before. There is nome dropped nametable update messages in kernel: Mar 22 10:34:34 mitchelltelctrl2 kernel: Dropping name table update (0) of {0, 0, 0} from <1.1.1> key=0 Mar 22 10:34:34 mitchelltelctrl2 kernel: Dropping name table update (0) of {0, 0, 0} from <1.1.1> key=0 Mar 22 10:34:34 mitchelltelctrl2 kernel: Dropping name table update (0) of {0, 0, 0} from <1.1.1> key=0 Mar 22 10:34:34 mitchelltelctrl2 kernel: Dropping name table update (0) of {0, 0, 0} from <1.1.1> key=0 Mar 22 10:34:34 mitchelltelctrl2 kernel: Dropping name table update (0) of {0, 0, 0} from <1.1.1> key=0 Mar 22 10:34:34 mitchelltelctrl2 kernel: Dropping name table update (0) of {0, 0, 0} from <1.1.1> key=0 Mar 22 10:34:34 mitchelltelctrl2 kernel: Dropping name table update (0) of {0, 0, 0} from <1.1.1> key=0 Mar 22 10:34:34 mitchelltelctrl2 kernel: Dropping name table update (0) of {0, 0, 0} from <1.1.1> key=0 Mar 22 10:34:34 mitchelltelctrl2 kernel: Dropping name table update (0) of {0, 0, 0} from <1.1.1> key=0 Mar 22 10:34:34 mitchelltelctrl2 kernel: Dropping name table update (0) of {0, 0, 16600} from <1.1.1> key=4294915584 but they do not mention port 104. If I restart the application on 1 having 104:1025 open, it shows up on 2. -----Original Message----- From: Rune Torgersen [mailto:ru...@innovsys.com<mailto:ru...@innovsys.com>] Sent: Monday, March 21, 2016 12:17 AM To: Jon Maloy; Erik Hugne Cc: tipc-discussion@lists.sourceforge.net<mailto:tipc-discussion@lists.sourceforge.net> Subject: Re: [tipc-discussion] tipc nametable update problem Using TIPC_CLUSTER_SCOPE will work. This was new system bring-up, and code was ported from older system, which used TIPC 1.7.7 driver. A quick search and replace of TIPC_ZONE_SCOPE is not a bad workaround. ________________________________________ From: Jon Maloy [jon.ma...@ericsson.com<mailto:jon.ma...@ericsson.com>] Sent: Saturday, March 19, 2016 10:57 AM To: Erik Hugne Cc: tipc-discussion@lists.sourceforge.net<mailto:tipc-discussion@lists.sourceforge.net> Subject: Re: [tipc-discussion] tipc nametable update problem Maybe not completely trivial, but not very complex either. I know I failed to describe this verbally to you at one moment, but I can put it on paper, and you will realize it is not a big deal. If you or anybody else are interested I can make an effort to describe this next week. I don't have time to implement it myself at the moment. ///jon From: Erik Hugne [mailto:erik.hu...@gmail.com<mailto:erik.hu...@gmail.com>] Sent: Friday, 18 March, 2016 12:38 To: Jon Maloy Subject: RE: [tipc-discussion] tipc nametable update problem Agree. But implementing a new lookup mechanism is not trivial.. :) @Rune afaik there is no functional limitation on using cluster scoped publications, so i hope that's an acceptable workaround for you. //E On Mar 18, 2016 16:46, "Jon Maloy" <jon.ma...@ericsson.com<mailto:jon.ma...@ericsson.com><mailto:jon.ma...@ericsson.com<mailto:jon.ma...@ericsson.com>>> wrote: Still weird that this starts happening now, when this issue is supposed to be remedied, and not earlier, when it wasn't. We really need that "permit overlapping publications" solution I have been preaching about. Br ///jon > -----Original Message----- > From: Rune Torgersen > [mailto:ru...@innovsys.com<mailto:ru...@innovsys.com><mailto:ru...@innovsys.com<mailto:ru...@innovsys.com>>] > Sent: Friday, 18 March, 2016 10:25 > To: 'Erik Hugne' > Cc: > tipc-discussion@lists.sourceforge.net<mailto:tipc-discussion@lists.sourceforge.net><mailto:tipc-discussion@lists.sourceforge.net<mailto:tipc-discussion@lists.sourceforge.net>> > Subject: Re: [tipc-discussion] tipc nametable update problem > > Yes I have. > There are quite a few at the same time like this: > > Mar 17 20:08:58 restarttv kernel: Dropping name table update (0) of > {1853110816, > 1952998688, 1801810542} from <1.1.1> key=1633905523 > Mar 17 20:08:58 restarttv kernel: Dropping name table update (0) of > {542000723, > 544613732, 544437616} from <1.1.1> key=167800175 > Mar 17 20:08:58 restarttv kernel: Dropping name table update (0) of > {544239474, > 1953325424, 543582572} from <1.1.1> key=1930035237 > Mar 17 20:08:58 restarttv kernel: Dropping name table update (0) of > {1933189232, > 1869771885, 1634738291} from <1.1.1> key=1768843040 > Mar 17 20:08:58 restarttv kernel: Dropping name table update (0) of > {1717660012, > 1701054976, 628308512} from <1.1.1> key=1869881446 > Mar 17 20:08:58 restarttv kernel: Dropping name table update (0) of {16397, > 1073741824, 16397} from <1.1.1> key=29285 > Mar 17 20:08:58 restarttv kernel: Dropping name table update (0) of > {1633943667, > 1752134260, 544367969} from <1.1.1> key=1679834144 > Mar 17 20:08:58 restarttv kernel: Dropping name table update (0) of > {1869771808, > 2003986804, 1698300018} from <1.1.1> key=4294915584 > Mar 17 20:08:58 restarttv kernel: Dropping name table update (0) of > {1073741824, > 65279, 4294902016} from <1.1.1> key=1073741824 > Mar 17 20:08:58 restarttv kernel: Dropping name table update (0) of {65279, > 4294901760, 59154} from <1.1.1> key=65023 > > > From: Erik Hugne > [mailto:erik.hu...@gmail.com<mailto:erik.hu...@gmail.com><mailto:erik.hu...@gmail.com<mailto:erik.hu...@gmail.com>>] > Sent: Friday, March 18, 2016 1:48 AM > To: Rune Torgersen > Cc: > tipc-discussion@lists.sourceforge.net<mailto:tipc-discussion@lists.sourceforge.net><mailto:tipc-discussion@lists.sourceforge.net<mailto:tipc-discussion@lists.sourceforge.net>> > Subject: Re: [tipc-discussion] tipc nametable update problem > > > Hi Rune. > When the problem occurs, have you seen any traces like "tipc: Dropping name > table update...." ? > > //E > On Mar 18, 2016 02:11, "Rune Torgersen" > <ru...@innovsys.com<mailto:ru...@innovsys.com><mailto:ru...@innovsys.com<mailto:ru...@innovsys.com>><mailto:ru...@innovsys.com<mailto:ru...@innovsys.com><mailto:ru...@innovsys.com<mailto:ru...@innovsys.com>>>> > wrote: > More info. > The failing ports are all opened as TIPC_ZONE_SCOPE. > Addresses of the two computers are 1.1.1 and 1.1.2. > > If I change the open param to TIPC_CLUSTER_SCOPE, the nametable seems to > update correctly. > > > -----Original Message----- > From: Rune Torgersen > [mailto:ru...@innovsys.com<mailto:ru...@innovsys.com><mailto:ru...@innovsys.com<mailto:ru...@innovsys.com>><mailto:ru...@innovsys.com<mailto:ru...@innovsys.com><mailto:ru...@innovsys.com<mailto:ru...@innovsys.com>>>] > Sent: Thursday, March 17, 2016 7:06 PM > To: > 'tipc-discussion@lists.sourceforge.net<mailto:tipc-discussion@lists.sourceforge.net><mailto:tipc-discussion@lists.sourceforge.net<mailto:tipc-discussion@lists.sourceforge.net>><mailto:tipc-<mailto:tipc-><mailto:tipc-<mailto:tipc->> > discuss...@lists.sourceforge.net<mailto:discuss...@lists.sourceforge.net><mailto:discuss...@lists.sourceforge.net<mailto:discuss...@lists.sourceforge.net>>>' > Subject: [tipc-discussion] tipc nametable update problem > > Hi. > > The product I work on uses TIPC for communication between different > computers on a network. We've actually been using older version (1.7.7 and > older > ) for nearly 10 years. > > On a new product, we're using the latest Ubuntu server (16.04, still in beta) > using > kernel 4.4.0. > > On several occasions now, after boot, programs that open TIPC sockets during > the boot process, have ports that does not show in the nametable on the other > computer. This of course causes the programs to not being able to talk. > If we restart the program, reopening the TIPC port, then it shows up on both > sides. > > > I know this is somewhat sparse info, but I am not sure where to start to look > at > this. > > One piece of info that might be useful, is that we kind of require the old > interface > naming on our interfaces, so we have turned off systemd's ethernet naming > scheme, and use udev to name the devices. > > This should be done well before we initializer the tipc driver module and > give it a > netid and address and enable the bearer links. > > ------------------------------------------------------------------------------ > Transform Data into Opportunity. > Accelerate data analysis in your applications with > Intel Data Analytics Acceleration Library. > Click to learn more. > http://pubads.g.doubleclick.net/gampad/clk?id=278785231&iu=/4140 > _______________________________________________ > tipc-discussion mailing list > tipc-discussion@lists.sourceforge.net<mailto:tipc-discussion@lists.sourceforge.net><mailto:tipc-discussion@lists.sourceforge.net<mailto:tipc-discussion@lists.sourceforge.net>><mailto:tipc-<mailto:tipc-><mailto:tipc-<mailto:tipc->> > discuss...@lists.sourceforge.net<mailto:discuss...@lists.sourceforge.net><mailto:discuss...@lists.sourceforge.net<mailto:discuss...@lists.sourceforge.net>>> > https://lists.sourceforge.net/lists/listinfo/tipc-discussion > > ------------------------------------------------------------------------------ > Transform Data into Opportunity. > Accelerate data analysis in your applications with > Intel Data Analytics Acceleration Library. > Click to learn more. > http://pubads.g.doubleclick.net/gampad/clk?id=278785231&iu=/4140 > _______________________________________________ > tipc-discussion mailing list > tipc-discussion@lists.sourceforge.net<mailto:tipc-discussion@lists.sourceforge.net><mailto:tipc-discussion@lists.sourceforge.net<mailto:tipc-discussion@lists.sourceforge.net>><mailto:tipc-<mailto:tipc-><mailto:tipc-<mailto:tipc->> > discuss...@lists.sourceforge.net<mailto:discuss...@lists.sourceforge.net><mailto:discuss...@lists.sourceforge.net<mailto:discuss...@lists.sourceforge.net>>> > https://lists.sourceforge.net/lists/listinfo/tipc-discussion > ------------------------------------------------------------------------------ > Transform Data into Opportunity. > Accelerate data analysis in your applications with > Intel Data Analytics Acceleration Library. > Click to learn more. > http://pubads.g.doubleclick.net/gampad/clk?id=278785231&iu=/4140 > _______________________________________________ > tipc-discussion mailing list > tipc-discussion@lists.sourceforge.net<mailto:tipc-discussion@lists.sourceforge.net><mailto:tipc-discussion@lists.sourceforge.net<mailto:tipc-discussion@lists.sourceforge.net>> > https://lists.sourceforge.net/lists/listinfo/tipc-discussion ------------------------------------------------------------------------------ Transform Data into Opportunity. Accelerate data analysis in your applications with Intel Data Analytics Acceleration Library. Click to learn more. http://pubads.g.doubleclick.net/gampad/clk?id=278785231&iu=/4140 _______________________________________________ tipc-discussion mailing list tipc-discussion@lists.sourceforge.net<mailto:tipc-discussion@lists.sourceforge.net> https://lists.sourceforge.net/lists/listinfo/tipc-discussion ------------------------------------------------------------------------------ Transform Data into Opportunity. Accelerate data analysis in your applications with Intel Data Analytics Acceleration Library. Click to learn more. http://pubads.g.doubleclick.net/gampad/clk?id=278785351&iu=/4140 _______________________________________________ tipc-discussion mailing list tipc-discussion@lists.sourceforge.net<mailto:tipc-discussion@lists.sourceforge.net> https://lists.sourceforge.net/lists/listinfo/tipc-discussion ------------------------------------------------------------------------------ Transform Data into Opportunity. Accelerate data analysis in your applications with Intel Data Analytics Acceleration Library. Click to learn more. http://pubads.g.doubleclick.net/gampad/clk?id=278785351&iu=/4140 _______________________________________________ tipc-discussion mailing list tipc-discussion@lists.sourceforge.net<mailto:tipc-discussion@lists.sourceforge.net> https://lists.sourceforge.net/lists/listinfo/tipc-discussion ------------------------------------------------------------------------------ Transform Data into Opportunity. Accelerate data analysis in your applications with Intel Data Analytics Acceleration Library. Click to learn more. http://pubads.g.doubleclick.net/gampad/clk?id=278785351&iu=/4140 _______________________________________________ tipc-discussion mailing list tipc-discussion@lists.sourceforge.net<mailto:tipc-discussion@lists.sourceforge.net> https://lists.sourceforge.net/lists/listinfo/tipc-discussion ------------------------------------------------------------------------------ Transform Data into Opportunity. Accelerate data analysis in your applications with Intel Data Analytics Acceleration Library. Click to learn more. http://pubads.g.doubleclick.net/gampad/clk?id=278785351&iu=/4140 _______________________________________________ tipc-discussion mailing list tipc-discussion@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/tipc-discussion ------------------------------------------------------------------------------ Transform Data into Opportunity. Accelerate data analysis in your applications with Intel Data Analytics Acceleration Library. Click to learn more. http://pubads.g.doubleclick.net/gampad/clk?id=278785351&iu=/4140 _______________________________________________ tipc-discussion mailing list tipc-discussion@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/tipc-discussion
