I'll take your advice and give it a shot, thanks :)

2015.06.12. 14:43 keltezéssel, Christopher Forgeron írta:
Ah, but the 'why' will come later, after we know for sure what the 'what' is in your problem.

I'm just pointing out the problems that I'm having, as yours sound similar. Once the box runs out of memory, all sorts of interesting things can happen. Perhaps that's not your case, but it's quite possible.

Setup a remote terminal, do the copy again, and send in the last few lines of 'vmstat 5' after it's locked up, perhaps I can help.

On Fri, Jun 12, 2015 at 9:39 AM, Cs <bim...@field.hu <mailto:bim...@field.hu>> wrote:

    but why is that machine runs fine except the network if it's
    memory related? swap didn't increased before the network outage.


    2015.06.12. 14:37 keltezéssel, Christopher Forgeron írta:
    rsycn burns memory - I'd say you have a good chance you're
    running out of mem before it's replenished.

    For vmstat 5 - Don't run it on console. Connect via a second box
    with ssh, and run it there - That way it's the last thing on the
    ssh terminal screen when the box dies, and you'll have your proof.

    On Fri, Jun 12, 2015 at 9:31 AM, Cs <bim...@field.hu
    <mailto:bim...@field.hu>> wrote:

        machine has been restarted before I could check the "vmstat
        5" output. Yep, it's rsync. Anyway I disabled the backup
        transfer it'll solve, but I can't really accept this for
        solution.


        2015.06.12. 14 <tel:2015.06.12.%2014>:29 keltezéssel,
        Christopher Forgeron írta:

            Well, even at low speed it could drop due to memory from
            what I've seen.

            What was the last line from vmstat 5 before it locked up?

              I find that the em driver isn't crap, but there is a
            deeper problem inside
            of FreeBSD that is being exposed now - For me it's due to
            faster network
            connections.

              Are you using rsync to move the files?

            On Fri, Jun 12, 2015 at 7:17 AM, Cs <bim...@field.hu
            <mailto:bim...@field.hu>> wrote:

                it seems it's not memory related. Server just died a
                few minutes ago
                during transporting the backup (400GB) around 800Mbps
                speed..
                will disable remote backup, it's a shame that em
                driver is such a crap.


                2015.06.08. 5:01 keltezéssel, Christopher Forgeron írta:

                    You know what helped me:

                    'vmstat 5'

                    Leave that running. If the last thing on the
                    console after a crash/hang is
                    vmstat showing 8k of memory left, then you're in
                    the same problem-park as
                    me.

                    My 10.1 96GiB RAM box is chewing ~8 GiB of RAM in
                    less than 5 seconds, and
                    then crashing/panicking/hanging.

                    There's others with this issues if you search for
                    it; a sysctl
                    to vm.v_free_min to double or triple that value
                    may help, but first let us
                    know if that's what is bonking your sever.



                    On Sun, Jun 7, 2015 at 11:03 AM, Cs
                    <bim...@field.hu <mailto:bim...@field.hu>> wrote:

                      ok, just lowered it to 1500 but please also
                    note that it was on 1500 for

                        2
                        years

                        2015.06.07. 14 <tel:2015.06.07.%2014>:57
                        keltezéssel, Rick Macklem írta:

                          Since disabling TSO didn't help, you could
                        try dropping to 1500mtu

                            on both interfaces. Some people run into
                            problems when 9K jumbo clusters
                            fragment the kernel address space used to
                            allocate mbufs.

                            Good luck with it, rick

                            ----- Original Message -----

                              Hi All,

                                It worked fine for two weeks but I
                                had a network outage 2 days ago
                                then
                                today. Tried to disable rxcsum and
                                txcsum after the first one, didn't
                                help. Don't know what else to do it's
                                a shame that I can't use this
                                card
                                with fbsd i REALLY don't want to
                                install linux instead but my
                                production
                                servers outages are not welcomed by
                                the customers..

                                2015.05.26. 10
                                <tel:2015.05.26.%2010>:36
                                keltezéssel, Cs írta:

                                  Thanks Mark, good idea. I found
                                this thread which is exactly the

                                    same
                                    problem as mine:


                                    
https://forums.freebsd.org/threads/workaround-freebsd-10-1-sudden-network-down.49264/

                                    Will see if it helps in a couple
                                    weeks.

                                    Regards,
                                    Csaba

                                    2015.05.26. 10
                                    <tel:2015.05.26.%2010>:30
                                    keltezéssel, Mark Schouten írta:

                                      Oh, didn't see your lowest
                                    remark. Then, the next thing that
                                    comes

                                        past here a few times per
                                        week is 'Try disabling TSO'.


                                        Met vriendelijke groeten,

                                        --
                                        Kerio Operator in de Cloud?
                                        https://www.kerioindecloud.nl/
                                        Mark Schouten  | Tuxis
                                        Internet Engineering
                                        KvK: 61527076 |
                                        http://www.tuxis.nl/
                                        T: 0318 200208 |
                                        i...@tuxis.nl
                                        <mailto:i...@tuxis.nl>



                                             Van:   Cs
                                        <bim...@field.hu
                                        <mailto:bim...@field.hu>>
                                             Aan:   Mark Schouten
                                        <m...@tuxis.nl
                                        <mailto:m...@tuxis.nl>>
Cc: <freebsd-net@freebsd.org
                                        <mailto:freebsd-net@freebsd.org>>
                                             Verzonden:  25-5-2015 11:12
                                             Onderwerp:   Re: FreeBSD
                                        10.1-REL - network
                                        unaccessible after
                                             high
                                        traffic

                                        It was on 1500 for ~3 years :)
                                             Regards,
                                        Csaba
                                                 On May 25, 2015,
                                        10:30, at 10:30, Mark Schouten
                                                 <m...@tuxis.nl
                                        <mailto:m...@tuxis.nl>>
                                        wrote:

                                          Try lowering your mtu to
                                        1500, that worked miracles
                                        for me..

                                            --
                                            Mark Schouten
                                            Tuxis Internet Engineering
                                            m...@tuxis.nl
                                            <mailto:m...@tuxis.nl> /
                                            0318 200208

                                               On 25 May 2015, at
                                            09:36, "Cs"
                                            <bim...@field.hu
                                            <mailto:bim...@field.hu>>
                                            wrote:

                                                     Hi all,
                                                     I have two
                                                FreeBSd 10.1-RELEASE
                                                servers connected to each
                                                     other.
                                                They

                                                  were connected via
                                                cross link, but they
                                                are connected to a cisco

                                            switch
                                            now (the problem was the
                                            same with cross link
                                            too). When
                                            transferring
                                            huge files (50-500GB
                                            backup files) via Gigabit
                                            (it is important!)
                                            the
                                            network randomly dies.
                                            The backup runs every
                                            day/week and
                                            sometimes the
                                            connection is ok for
                                            months sometimes it
                                            happens twice a week.
                                            When the
                                            network dies I can log in
                                            to the server via IPMI
                                            and use the
                                            console
                                            everything is OK, but
                                            can't send anything out
                                            on the network.
                                            ifconfig
                                            em0 down/up doesn't help
                                            nor netif restart. The
                                            problem never
                                            occured
                                            when I used 100Mbit
                                            connection between them,
                                            but it was 3com NIC
                                            (xl),
                                            gigabit adapter is Intel
                                            (em0). When I limit the
                                            transfer rate
                                            (rsync
                                            bandwith limit or ipfw
                                            pipe) the problem is much
                                            more rare.

                                                  I tried to set
                                            these tuning parameters
                                            on both servers with

                                                different

                                                  buffer size but
                                                nothing helped:

                                                  # cat /etc/sysctl.conf

                                                security.bsd.see_other_uids=0
                                                net.inet.tcp.recvspace=512000
                                                net.route.netisr_maxqlen=2048
                                                kern.ipc.nmbclusters=1310720
                                                
net.inet.tcp.sendbuf_max=16777216
                                                
net.inet.tcp.recvbuf_max=16777216
                                                kern.ipc.soacceptqueue=32768
                                                     # cat
                                                /boot/loader.conf
                                                geom_mirror_load="YES" #
                                                RAID1 disk driver
                                                (see gmirror(8))
                                                ipfw_load="YES"
                                                
net.inet.ip.fw.default_to_accept=1
                                                kern.maxusers=4096
                                                accf_data_load="YES"
                                                     The duplex
                                                settings are
                                                identical on both
                                                servers.
                                                     Server A:
                                                em1:
                                                
flags=8843<UP,BROADCAST,RUNNING,SIMPLEX,MULTICAST>
                                                metric 0
                                                mtu

                                                  9000


                                            
options=4219b<RXCSUM,TXCSUM,VLAN_MTU,VLAN_HWTAGGING,VLAN_HWCSUM,TSO4,WOL_MAGIC,VLAN_HWTSO>


                                                        ether
                                            00:25:90:24:52:66

                                                           inet
                                                x.x.x.x netmask
                                                0xfffffe00 broadcast
                                                x.x.x.x
                                                           nd6
                                                
options=29<PERFORMNUD,IFDISABLED,AUTO_LINKLOCAL>
                                                           media:
                                                Ethernet autoselect
                                                (1000baseT <full-duplex>)
                                                           status: active
                                                     Server B:
                                                em0:
                                                
flags=8843<UP,BROADCAST,RUNNING,SIMPLEX,MULTICAST>
                                                metric 0
                                                mtu

                                                  9000


                                            
options=4219b<RXCSUM,TXCSUM,VLAN_MTU,VLAN_HWTAGGING,VLAN_HWCSUM,TSO4,WOL_MAGIC,VLAN_HWTSO>


                                                        ether
                                            00:30:48:dd:fe:3e

                                                           inet
                                                x.x.x.x netmask
                                                0xfffffe00 broadcast
                                                x.x.x.x
                                                           nd6
                                                
options=29<PERFORMNUD,IFDISABLED,AUTO_LINKLOCAL>
                                                           media:
                                                Ethernet autoselect
                                                (1000baseT <full-duplex>)
                                                           status: active
                                                     Today I tried to
                                                set mtu to 9000 but
                                                in tcpdump I see that
                                                     during
                                                scp

                                                  it is still 1500:

                                                    x.x.x.x.222 >
                                            x.x.x.x.37612: Flags [.],
                                            cksum 0xb6ee

                                                       (incorrect ->

                                                  0xda6f), seq 35749,
                                                ack 113701596, win
                                                7986, options [nop,nop,TS

                                            val
                                            3103966325
                                            <tel:3103966325> ecr
                                            853712893], length 0

                                              09:27:33.912354 IP (tos
                                            0x8, ttl 64, id 1028,
                                            offset 0, flags

                                                [DF],

                                                  proto TCP (6),
                                                length 1500)

                                              09:27:33.912358 IP (tos
                                            0x8, ttl 64, id 1029,
                                            offset 0, flags

                                                [DF],

                                                  proto TCP (6),
                                                length 1500)

                                                    Any ideas? Thanks
                                            guys!

                                                
_______________________________________________
                                                freebsd-net@freebsd.org
                                                <mailto:freebsd-net@freebsd.org>
                                                mailing list
                                                
http://lists.freebsd.org/mailman/listinfo/freebsd-net
                                                To unsubscribe, send
                                                any mail to

"freebsd-net-unsubscr...@freebsd.org
                                                
<mailto:freebsd-net-unsubscr...@freebsd.org>"

                                            
_______________________________________________

                                        freebsd-net@freebsd.org
                                        <mailto:freebsd-net@freebsd.org>
                                        mailing list
                                        
http://lists.freebsd.org/mailman/listinfo/freebsd-net
                                        To unsubscribe, send any mail to
                                        "freebsd-net-unsubscr...@freebsd.org
                                        
<mailto:freebsd-net-unsubscr...@freebsd.org>"


                                         
_______________________________________________

                                    freebsd-net@freebsd.org
                                    <mailto:freebsd-net@freebsd.org>
                                    mailing list
                                    
http://lists.freebsd.org/mailman/listinfo/freebsd-net
                                    To unsubscribe, send any mail to
                                    "freebsd-net-unsubscr...@freebsd.org
                                    
<mailto:freebsd-net-unsubscr...@freebsd.org>"

                                    
_______________________________________________

                                freebsd-net@freebsd.org
                                <mailto:freebsd-net@freebsd.org>
                                mailing list
                                
http://lists.freebsd.org/mailman/listinfo/freebsd-net
                                To unsubscribe, send any mail to
                                "freebsd-net-unsubscr...@freebsd.org
                                <mailto:freebsd-net-unsubscr...@freebsd.org>"

                                _______________________________________________

                        freebsd-net@freebsd.org
                        <mailto:freebsd-net@freebsd.org> mailing list
                        http://lists.freebsd.org/mailman/listinfo/freebsd-net
                        To unsubscribe, send any mail to
                        "freebsd-net-unsubscr...@freebsd.org
                        <mailto:freebsd-net-unsubscr...@freebsd.org>"

                        _______________________________________________

                    freebsd-net@freebsd.org
                    <mailto:freebsd-net@freebsd.org> mailing list
                    http://lists.freebsd.org/mailman/listinfo/freebsd-net
                    To unsubscribe, send any mail to
                    "freebsd-net-unsubscr...@freebsd.org
                    <mailto:freebsd-net-unsubscr...@freebsd.org>"

                _______________________________________________
                freebsd-net@freebsd.org
                <mailto:freebsd-net@freebsd.org> mailing list
                http://lists.freebsd.org/mailman/listinfo/freebsd-net
                To unsubscribe, send any mail to
                "freebsd-net-unsubscr...@freebsd.org
                <mailto:freebsd-net-unsubscr...@freebsd.org>"

            _______________________________________________
            freebsd-net@freebsd.org <mailto:freebsd-net@freebsd.org>
            mailing list
            http://lists.freebsd.org/mailman/listinfo/freebsd-net
            To unsubscribe, send any mail to
            "freebsd-net-unsubscr...@freebsd.org
            <mailto:freebsd-net-unsubscr...@freebsd.org>"


        _______________________________________________
        freebsd-net@freebsd.org <mailto:freebsd-net@freebsd.org>
        mailing list
        http://lists.freebsd.org/mailman/listinfo/freebsd-net
        To unsubscribe, send any mail to
        "freebsd-net-unsubscr...@freebsd.org
        <mailto:freebsd-net-unsubscr...@freebsd.org>"





_______________________________________________
freebsd-net@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-net
To unsubscribe, send any mail to "freebsd-net-unsubscr...@freebsd.org"

Reply via email to