[ https://issues.apache.org/jira/browse/KAFKA-5007?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15958944#comment-15958944 ]
Joseph Aliase commented on KAFKA-5007: -------------------------------------- [~junrao] Thanks for responding back. I do see sockets in close wait but they are not causing Open files to fill up. Instead, I see lot of below lines in lsof command. This is what is making application hit its open file limit. java 24739 cyclone 696u unix 0xffff880035b35000 0t0 42713601 socket java 24739 cyclone 697u unix 0xffff880035b35000 0t0 42713601 socket java 24739 cyclone 698u unix 0xffff880035b35000 0t0 42713601 socket java 24739 cyclone 699u unix 0xffff880035b35000 0t0 42713601 socket java 24739 cyclone 700u unix 0xffff880035b35000 0t0 42713601 socket java 24739 cyclone 701u unix 0xffff880035b35000 0t0 42713601 socket java 24739 cyclone 702u unix 0xffff880035b35000 0t0 42713601 socket java 24739 cyclone 703u unix 0xffff880035b35000 0t0 42713601 socket java 24739 cyclone 704u unix 0xffff880035b35000 0t0 42713601 socket java 24739 cyclone 705u unix 0xffff880035b35000 0t0 42713601 socket java 24739 cyclone 706u unix 0xffff880035b35000 0t0 42713601 socket java 24739 cyclone 707u unix 0xffff880035b35000 0t0 42713601 socket java 24739 cyclone 708u unix 0xffff880035b35000 0t0 42713601 socket java 24739 cyclone 709u unix 0xffff880035b35000 0t0 42713601 socket java 24739 cyclone 710u unix 0xffff880035b35000 0t0 42713601 socket java 24739 cyclone 711u unix 0xffff880035b35000 0t0 42713601 socket java 24739 cyclone 712u unix 0xffff880035b35000 0t0 42713601 socket java 24739 cyclone 713u unix 0xffff880035b35000 0t0 42713601 socket java 24739 cyclone 714u unix 0xffff880035b35000 0t0 42713601 socket java 24739 cyclone 715u unix 0xffff880035b35000 0t0 42713601 socket java 24739 cyclone 716u unix 0xffff880035b35000 0t0 42713601 socket java 24739 cyclone 717u unix 0xffff880035b35000 0t0 42713601 socket java 24739 cyclone 718u unix 0xffff880035b35000 0t0 42713601 socket java 24739 cyclone 719u unix 0xffff880035b35000 0t0 42713601 socket java 24739 cyclone 720u unix 0xffff880035b35000 0t0 42713601 socket java 24739 cyclone 721u unix 0xffff880035b35000 0t0 42713601 socket java 24739 cyclone 722u unix 0xffff880035b35000 0t0 42713601 socket java 24739 cyclone 723u unix 0xffff880035b35000 0t0 42713601 socket java 24739 cyclone 724u unix 0xffff880035b35000 0t0 42713601 socket java 24739 cyclone 725u unix 0xffff880035b35000 0t0 42713601 socket java 24739 cyclone 726u unix 0xffff880035b35000 0t0 42713601 socket java 24739 cyclone 727u unix 0xffff880035b35000 0t0 42713601 socket java 24739 cyclone 728u unix 0xffff880035b35000 0t0 42713601 socket java 24739 cyclone 729u unix 0xffff880035b35000 0t0 42713601 socket java 24739 cyclone 730u unix 0xffff880035b35000 0t0 42713601 socket java 24739 cyclone 731u unix 0xffff880035b35000 0t0 42713601 socket java 24739 cyclone 732u unix 0xffff880035b35000 0t0 42713601 socket java 24739 cyclone 733u unix 0xffff880035b35000 0t0 42713601 socket java 24739 cyclone 734u unix 0xffff880035b35000 0t0 42713601 socket java 24739 cyclone 735u unix 0xffff880035b35000 0t0 42713601 socket java 24739 cyclone 736u unix 0xffff880035b35000 0t0 42713601 socket java 24739 cyclone 737u unix 0xffff880035b35000 0t0 42713601 socket java 24739 cyclone 738u unix 0xffff880035b35000 0t0 42713601 socket java 24739 cyclone 739u unix 0xffff880035b35000 0t0 42713601 socket java 24739 cyclone 740u unix 0xffff880035b35000 0t0 42713601 socket java 24739 cyclone 741u unix 0xffff880035b35000 0t0 42713601 socket java 24739 cyclone 742u unix 0xffff880035b35000 0t0 42713601 socket java 24739 cyclone 743u unix 0xffff880035b35000 0t0 42713601 socket java 24739 cyclone 744u unix 0xffff880035b35000 0t0 42713601 socket java 24739 cyclone 745u unix 0xffff880035b35000 0t0 42713601 socket java 24739 cyclone 746u unix 0xffff880035b35000 0t0 42713601 socket java 24739 cyclone 747u unix 0xffff880035b35000 0t0 42713601 socket java 24739 cyclone 748u unix 0xffff880035b35000 0t0 42713601 socket java 24739 cyclone 749u unix 0xffff880035b35000 0t0 42713601 socket java 24739 cyclone 750u unix 0xffff880035b35000 0t0 42713601 socket java 24739 cyclone 751u unix 0xffff880035b35000 0t0 42713601 socket java 24739 cyclone 752u unix 0xffff880035b35000 0t0 42713601 socket java 24739 cyclone 753u unix 0xffff880035b35000 0t0 42713601 socket java 24739 cyclone 754u unix 0xffff880035b35000 0t0 42713601 socket java 24739 cyclone 755u unix 0xffff880035b35000 0t0 42713601 socket java 24739 cyclone 756u unix 0xffff880035b35000 0t0 42713601 socket java 24739 cyclone 757u unix 0xffff880035b35000 0t0 42713601 socket java 24739 cyclone 758u unix 0xffff880035b35000 0t0 42713601 socket java 24739 cyclone 759u unix 0xffff880035b35000 0t0 42713601 socket java 24739 cyclone 760u unix 0xffff880035b35000 0t0 42713601 socket java 24739 cyclone 761u unix 0xffff880035b35000 0t0 42713601 socket java 24739 cyclone 762u unix 0xffff880035b35000 0t0 42713601 socket java 24739 cyclone 763u unix 0xffff880035b35000 0t0 42713601 socket java 24739 cyclone 764u unix 0xffff880035b35000 0t0 42713601 socket java 24739 cyclone 765u unix 0xffff880035b35000 0t0 42713601 socket java 24739 cyclone 766u unix 0xffff880035b35000 0t0 42713601 socket java 24739 cyclone 767u unix 0xffff880035b35000 0t0 42713601 socket java 24739 cyclone 768u unix 0xffff880035b35000 0t0 42713601 socket java 24739 cyclone 769u unix 0xffff880035b35000 0t0 42713601 socket java 24739 cyclone 770u unix 0xffff880035b35000 0t0 42713601 socket java 24739 cyclone 771u unix 0xffff880035b35000 0t0 42713601 socket java 24739 cyclone 772u unix 0xffff880035b35000 0t0 42713601 socket java 24739 cyclone 773u unix 0xffff880035b35000 0t0 42713601 socket java 24739 cyclone 774u unix 0xffff880035b35000 0t0 42713601 socket java 24739 cyclone 775u unix 0xffff880035b35000 0t0 42713601 socket java 24739 cyclone 776u unix 0xffff880035b35000 0t0 42713601 socket java 24739 cyclone 777u unix 0xffff880035b35000 0t0 42713601 socket java 24739 cyclone 778u unix 0xffff880035b35000 0t0 42713601 socket java 24739 cyclone 779u unix 0xffff880035b35000 0t0 42713601 socket java 24739 cyclone 780u unix 0xffff880035b35000 0t0 42713601 socket java 24739 cyclone 781u unix 0xffff880035b35000 0t0 42713601 socket java 24739 cyclone 782u unix 0xffff880035b35000 0t0 42713601 socket java 24739 cyclone 783u unix 0xffff880035b35000 0t0 42713601 socket java 24739 cyclone 784u unix 0xffff880035b35000 0t0 42713601 socket java 24739 cyclone 785u unix 0xffff880035b35000 0t0 42713601 socket java 24739 cyclone 786u unix 0xffff880035b35000 0t0 42713601 socket java 24739 cyclone 787u unix 0xffff880035b35000 0t0 42713601 socket java 24739 cyclone 788u unix 0xffff880035b35000 0t0 42713601 socket java 24739 cyclone 789u unix 0xffff880035b35000 0t0 42713601 socket java 24739 cyclone 790u unix 0xffff880035b35000 0t0 42713601 socket java 24739 cyclone 791u unix 0xffff880035b35000 0t0 42713601 socket java 24739 cyclone 792u unix 0xffff880035b35000 0t0 42713601 socket java 24739 cyclone 793u unix 0xffff880035b35000 0t0 42713601 socket java 24739 cyclone 794u unix 0xffff880035b35000 0t0 42713601 socket java 24739 cyclone 795u unix 0xffff880035b35000 0t0 42713601 socket java 24739 cyclone 796u unix 0xffff880035b35000 0t0 42713601 socket java 24739 cyclone 797u unix 0xffff880035b35000 0t0 42713601 socket java 24739 cyclone 798u unix 0xffff880035b35000 0t0 42713601 socket java 24739 cyclone 799u unix 0xffff880035b35000 0t0 42713601 socket java 24739 cyclone 800u unix 0xffff880035b35000 0t0 42713601 socket java 24739 cyclone 801u unix 0xffff880035b35000 0t0 42713601 socket java 24739 cyclone 802u unix 0xffff880035b35000 0t0 42713601 socket java 24739 cyclone 803u unix 0xffff880035b35000 0t0 42713601 socket java 24739 cyclone 804u unix 0xffff880035b35000 0t0 42713601 socket java 24739 cyclone 805u unix 0xffff880035b35000 0t0 42713601 socket java 24739 cyclone 806u unix 0xffff880035b35000 0t0 42713601 socket java 24739 cyclone 807u unix 0xffff880035b35000 0t0 42713601 socket java 24739 cyclone 808u unix 0xffff880035b35000 0t0 42713601 socket java 24739 cyclone 809u unix 0xffff880035b35000 0t0 42713601 socket java 24739 cyclone 810u unix 0xffff880035b35000 0t0 42713601 socket java 24739 cyclone 811u unix 0xffff880035b35000 0t0 42713601 socket java 24739 cyclone 812u unix 0xffff880035b35000 0t0 42713601 socket java 24739 cyclone 813u unix 0xffff880035b35000 0t0 42713601 socket java 24739 cyclone 814u unix 0xffff880035b35000 0t0 42713601 socket java 24739 cyclone 815u unix 0xffff880035b35000 0t0 42713601 socket java 24739 cyclone 816u unix 0xffff880035b35000 0t0 42713601 socket java 24739 cyclone 817u unix 0xffff880035b35000 0t0 42713601 socket java 24739 cyclone 818u unix 0xffff880035b35000 0t0 42713601 socket java 24739 cyclone 819u unix 0xffff880035b35000 0t0 42713601 socket java 24739 cyclone 820u unix 0xffff880035b35000 0t0 42713601 socket java 24739 cyclone 821u unix 0xffff880035b35000 0t0 42713601 socket java 24739 cyclone 822u unix 0xffff880035b35000 0t0 42713601 socket java 24739 cyclone 823u unix 0xffff880035b35000 0t0 42713601 socket java 24739 cyclone 824u unix 0xffff880035b35000 0t0 42713601 socket java 24739 cyclone 825u unix 0xffff880035b35000 0t0 42713601 socket java 24739 cyclone 826u unix 0xffff880035b35000 0t0 42713601 socket java 24739 cyclone 827u unix 0xffff880035b35000 0t0 42713601 socket java 24739 cyclone 828u unix 0xffff880035b35000 0t0 42713601 socket java 24739 cyclone 829u unix 0xffff880035b35000 0t0 42713601 socket java 24739 cyclone 830u unix 0xffff880035b35000 0t0 42713601 socket java 24739 cyclone 831u unix 0xffff880035b35000 0t0 42713601 socket java 24739 cyclone 832u unix 0xffff880035b35000 0t0 42713601 socket java 24739 cyclone 833u unix 0xffff880035b35000 0t0 42713601 socket java 24739 cyclone 834u unix 0xffff880035b35000 0t0 42713601 socket java 24739 cyclone 835u unix 0xffff880035b35000 0t0 42713601 socket java 24739 cyclone 836u unix 0xffff880035b35000 0t0 42713601 socket java 24739 cyclone 837u unix 0xffff880035b35000 0t0 42713601 socket java 24739 cyclone 838u unix 0xffff880035b35000 0t0 42713601 socket java 24739 cyclone 839u unix 0xffff880035b35000 0t0 42713601 socket java 24739 cyclone 840u unix 0xffff880035b35000 0t0 42713601 socket java 24739 cyclone 841u unix 0xffff880035b35000 0t0 42713601 socket java 24739 cyclone 842u unix 0xffff880035b35000 0t0 42713601 socket java 24739 cyclone 843u unix 0xffff880035b35000 0t0 42713601 socket java 24739 cyclone 844u unix 0xffff880035b35000 0t0 42713601 socket java 24739 cyclone 845u unix 0xffff880035b35000 0t0 42713601 socket java 24739 cyclone 846u unix 0xffff880035b35000 0t0 42713601 socket java 24739 cyclone 847u unix 0xffff880035b35000 0t0 42713601 socket java 24739 cyclone 848u unix 0xffff880035b35000 0t0 42713601 socket java 24739 cyclone 849u unix 0xffff880035b35000 0t0 42713601 socket java 24739 cyclone 850u unix 0xffff880035b35000 0t0 42713601 socket java 24739 cyclone 851u unix 0xffff880035b35000 0t0 42713601 socket java 24739 cyclone 852u unix 0xffff880035b35000 0t0 42713601 socket java 24739 cyclone 853u unix 0xffff880035b35000 0t0 42713601 socket java 24739 cyclone 854u unix 0xffff880035b35000 0t0 42713601 socket java 24739 cyclone 855u unix 0xffff880035b35000 0t0 42713601 socket java 24739 cyclone 856u unix 0xffff880035b35000 0t0 42713601 socket java 24739 cyclone 857u unix 0xffff880035b35000 0t0 42713601 socket java 24739 cyclone 858u unix 0xffff880035b35000 0t0 42713601 socket java 24739 cyclone 859u unix 0xffff880035b35000 0t0 42713601 socket java 24739 cyclone 860u unix 0xffff880035b35000 0t0 42713601 socket java 24739 cyclone 861u unix 0xffff880035b35000 0t0 42713601 socket java 24739 cyclone 862u unix 0xffff880035b35000 0t0 42713601 socket java 24739 cyclone 863u unix 0xffff880035b35000 0t0 42713601 socket java 24739 cyclone 864u unix 0xffff880035b35000 0t0 42713601 socket java 24739 cyclone 865u unix 0xffff880035b35000 0t0 42713601 socket java 24739 cyclone 866u unix 0xffff880035b35000 0t0 42713601 socket java 24739 cyclone 867u unix 0xffff880035b35000 0t0 42713601 socket java 24739 cyclone 868u unix 0xffff880035b35000 0t0 42713601 socket java 24739 cyclone 869u unix 0xffff880035b35000 0t0 42713601 socket java 24739 cyclone 870u unix 0xffff880035b35000 0t0 42713601 socket java 24739 cyclone 871u unix 0xffff880035b35000 0t0 42713601 socket java 24739 cyclone 872u unix 0xffff880035b35000 0t0 42713601 socket java 24739 cyclone 873u unix 0xffff880035b35000 0t0 42713601 socket java 24739 cyclone 874u unix 0xffff880035b35000 0t0 42713601 socket java 24739 cyclone 875u unix 0xffff880035b35000 0t0 42713601 socket java 24739 cyclone 876u unix 0xffff880035b35000 0t0 42713601 socket java 24739 cyclone 877u unix 0xffff880035b35000 0t0 42713601 socket java 24739 cyclone 878u unix 0xffff880035b35000 0t0 42713601 socket java 24739 cyclone 879u unix 0xffff880035b35000 0t0 42713601 socket java 24739 cyclone 880u unix 0xffff880035b35000 0t0 42713601 socket java 24739 cyclone 881u unix 0xffff880035b35000 0t0 42713601 socket java 24739 cyclone 882u unix 0xffff880035b35000 0t0 42713601 socket java 24739 cyclone 883u unix 0xffff880035b35000 0t0 42713601 socket java 24739 cyclone 884u unix 0xffff880035b35000 0t0 42713601 socket java 24739 cyclone 885u unix 0xffff880035b35000 0t0 42713601 socket java 24739 cyclone 886u unix 0xffff880035b35000 0t0 42713601 socket java 24739 cyclone 887u unix 0xffff880035b35000 0t0 42713601 socket java 24739 cyclone 888u unix 0xffff880035b35000 0t0 42713601 socket java 24739 cyclone 889u unix 0xffff880035b35000 0t0 42713601 socket java 24739 cyclone 890u unix 0xffff880035b35000 0t0 42713601 socket java 24739 cyclone 891u unix 0xffff880035b35000 0t0 42713601 socket java 24739 cyclone 892u unix 0xffff880035b35000 0t0 42713601 socket java 24739 cyclone 893u unix 0xffff880035b35000 0t0 42713601 socket java 24739 cyclone 894u unix 0xffff880035b35000 0t0 42713601 socket java 24739 cyclone 895u unix 0xffff880035b35000 0t0 42713601 socket java 24739 cyclone 896u unix 0xffff880035b35000 0t0 42713601 socket java 24739 cyclone 897u unix 0xffff880035b35000 0t0 42713601 socket java 24739 cyclone 898u unix 0xffff880035b35000 0t0 42713601 socket java 24739 cyclone 899u unix 0xffff880035b35000 0t0 42713601 socket java 24739 cyclone 900u unix 0xffff880035b35000 0t0 42713601 socket java 24739 cyclone 901u unix 0xffff880035b35000 0t0 42713601 socket java 24739 cyclone 902u unix 0xffff880035b35000 0t0 42713601 socket java 24739 cyclone 903u unix 0xffff880035b35000 0t0 42713601 socket java 24739 cyclone 904u unix 0xffff880035b35000 0t0 42713601 socket java 24739 cyclone 905u unix 0xffff880035b35000 0t0 42713601 socket java 24739 cyclone 906u unix 0xffff880035b35000 0t0 42713601 socket java 24739 cyclone 907u unix 0xffff880035b35000 0t0 42713601 socket java 24739 cyclone 908u unix 0xffff880035b35000 0t0 42713601 socket java 24739 cyclone 909u unix 0xffff880035b35000 0t0 42713601 socket java 24739 cyclone 910u unix 0xffff880035b35000 0t0 42713601 socket java 24739 cyclone 911u unix 0xffff880035b35000 0t0 42713601 socket java 24739 cyclone 912u unix 0xffff880035b35000 0t0 42713601 socket java 24739 cyclone 913u unix 0xffff880035b35000 0t0 42713601 socket java 24739 cyclone 914u unix 0xffff880035b35000 0t0 42713601 socket java 24739 cyclone 915u unix 0xffff880035b35000 0t0 42713601 socket java 24739 cyclone 916u unix 0xffff880035b35000 0t0 42713601 socket java 24739 cyclone 917u unix 0xffff880035b35000 0t0 42713601 socket java 24739 cyclone 918u unix 0xffff880035b35000 0t0 42713601 socket java 24739 cyclone 919u unix 0xffff880035b35000 0t0 42713601 socket java 24739 cyclone 920u unix 0xffff880035b35000 0t0 42713601 socket java 24739 cyclone 921u unix 0xffff880035b35000 0t0 42713601 socket java 24739 cyclone 922u unix 0xffff880035b35000 0t0 42713601 socket java 24739 cyclone 923u unix 0xffff880035b35000 0t0 42713601 socket java 24739 cyclone 924u unix 0xffff880035b35000 0t0 42713601 socket java 24739 cyclone 925u unix 0xffff880035b35000 0t0 42713601 socket java 24739 cyclone 926u unix 0xffff880035b35000 0t0 42713601 socket java 24739 cyclone 927u unix 0xffff880035b35000 0t0 42713601 socket java 24739 cyclone 928u unix 0xffff880035b35000 0t0 42713601 socket java 24739 cyclone 929u unix 0xffff880035b35000 0t0 42713601 socket java 24739 cyclone 930u unix 0xffff880035b35000 0t0 42713601 socket java 24739 cyclone 931u unix 0xffff880035b35000 0t0 42713601 socket java 24739 cyclone 932u unix 0xffff880035b35000 0t0 42713601 socket java 24739 cyclone 933u unix 0xffff880035b35000 0t0 42713601 socket java 24739 cyclone 934u unix 0xffff880035b35000 0t0 42713601 socket java 24739 cyclone 935u unix 0xffff880035b35000 0t0 42713601 socket java 24739 cyclone 936u unix 0xffff880035b35000 0t0 42713601 socket java 24739 cyclone 937u unix 0xffff880035b35000 0t0 42713601 socket java 24739 cyclone 938u unix 0xffff880035b35000 0t0 42713601 socket java 24739 cyclone 939u unix 0xffff880035b35000 0t0 42713601 socket java 24739 cyclone 940u unix 0xffff880035b35000 0t0 42713601 socket java 24739 cyclone 941u unix 0xffff880035b35000 0t0 42713601 socket java 24739 cyclone 942u unix 0xffff880035b35000 0t0 42713601 socket java 24739 cyclone 943u unix 0xffff880035b35000 0t0 42713601 socket java 24739 cyclone 944u unix 0xffff880035b35000 0t0 42713601 socket java 24739 cyclone 945u unix 0xffff880035b35000 0t0 42713601 socket java 24739 cyclone 946u unix 0xffff880035b35000 0t0 42713601 socket java 24739 cyclone 947u unix 0xffff880035b35000 0t0 42713601 socket java 24739 cyclone 948u unix 0xffff880035b35000 0t0 42713601 socket java 24739 cyclone 949u unix 0xffff880035b35000 0t0 42713601 socket java 24739 cyclone 950u unix 0xffff880035b35000 0t0 42713601 socket java 24739 cyclone 951u unix 0xffff880035b35000 0t0 42713601 socket java 24739 cyclone 952u unix 0xffff880035b35000 0t0 42713601 socket java 24739 cyclone 953u unix 0xffff880035b35000 0t0 42713601 socket java 24739 cyclone 954u unix 0xffff880035b35000 0t0 42713601 socket java 24739 cyclone 955u unix 0xffff880035b35000 0t0 42713601 socket java 24739 cyclone 956u unix 0xffff880035b35000 0t0 42713601 socket java 24739 cyclone 957u unix 0xffff880035b35000 0t0 42713601 socket java 24739 cyclone 958u unix 0xffff880035b35000 0t0 42713601 socket java 24739 cyclone 959u unix 0xffff880035b35000 0t0 42713601 socket java 24739 cyclone 960u unix 0xffff880035b35000 0t0 42713601 socket java 24739 cyclone 961u unix 0xffff880035b35000 0t0 42713601 socket java 24739 cyclone 962u unix 0xffff880035b35000 0t0 42713601 socket java 24739 cyclone 963u unix 0xffff880035b35000 0t0 42713601 socket java 24739 cyclone 964u unix 0xffff880035b35000 0t0 42713601 socket java 24739 cyclone 965u unix 0xffff880035b35000 0t0 42713601 socket java 24739 cyclone 966u unix 0xffff880035b35000 0t0 42713601 socket java 24739 cyclone 967u unix 0xffff880035b35000 0t0 42713601 socket java 24739 cyclone 968u unix 0xffff880035b35000 0t0 42713601 socket java 24739 cyclone 969u unix 0xffff880035b35000 0t0 42713601 socket java 24739 cyclone 970u unix 0xffff880035b35000 0t0 42713601 socket java 24739 cyclone 971u unix 0xffff880035b35000 0t0 42713601 socket java 24739 cyclone 972u unix 0xffff880035b35000 0t0 42713601 socket java 24739 cyclone 973u unix 0xffff880035b35000 0t0 42713601 socket java 24739 cyclone 974u unix 0xffff880035b35000 0t0 42713601 socket java 24739 cyclone 975u unix 0xffff880035b35000 0t0 42713601 socket java 24739 cyclone 976u unix 0xffff880035b35000 0t0 42713601 socket java 24739 cyclone 977u unix 0xffff880035b35000 0t0 42713601 socket java 24739 cyclone 978u unix 0xffff880035b35000 0t0 42713601 socket java 24739 cyclone 979u unix 0xffff880035b35000 0t0 42713601 socket java 24739 cyclone 980u unix 0xffff880035b35000 0t0 42713601 socket java 24739 cyclone 981u unix 0xffff880035b35000 0t0 42713601 socket java 24739 cyclone 982u unix 0xffff880035b35000 0t0 42713601 socket java 24739 cyclone 983u unix 0xffff880035b35000 0t0 42713601 socket java 24739 cyclone 984u unix 0xffff880035b35000 0t0 42713601 socket java 24739 cyclone 985u unix 0xffff880035b35000 0t0 42713601 socket java 24739 cyclone 986u unix 0xffff880035b35000 0t0 42713601 socket java 24739 cyclone 987u unix 0xffff880035b35000 0t0 42713601 socket java 24739 cyclone 988u unix 0xffff880035b35000 0t0 42713601 socket java 24739 cyclone 989u unix 0xffff880035b35000 0t0 42713601 socket java 24739 cyclone 990u unix 0xffff880035b35000 0t0 42713601 socket java 24739 cyclone 991u unix 0xffff880035b35000 0t0 42713601 socket java 24739 cyclone 992u unix 0xffff880035b35000 0t0 42713601 socket java 24739 cyclone 993u unix 0xffff880035b35000 0t0 42713601 socket java 24739 cyclone 994u unix 0xffff880035b35000 0t0 42713601 socket java 24739 cyclone 995u unix 0xffff880035b35000 0t0 42713601 socket java 24739 cyclone 996u unix 0xffff880035b35000 0t0 42713601 socket java 24739 cyclone 997u unix 0xffff880035b35000 0t0 42713601 socket java 24739 cyclone 998u unix 0xffff880035b35000 0t0 42713601 socket java 24739 cyclone 999u unix 0xffff880035b35000 0t0 42713601 socket java 24739 cyclone 1000u unix 0xffff880035b35000 0t0 42713601 socket java 24739 cyclone 1001u unix 0xffff880035b35000 0t0 42713601 socket java 24739 cyclone 1002u unix 0xffff880035b35000 0t0 42713601 socket java 24739 cyclone 1003u unix 0xffff880035b35000 0t0 42713601 socket java 24739 cyclone 1004u unix 0xffff880035b35000 0t0 42713601 socket java 24739 cyclone 1005u unix 0xffff880035b35000 0t0 42713601 socket java 24739 cyclone 1006u unix 0xffff880035b35000 0t0 42713601 socket java 24739 cyclone 1007u unix 0xffff880035b35000 0t0 42713601 socket java 24739 cyclone 1008u unix 0xffff880035b35000 0t0 42713601 socket java 24739 cyclone 1009u unix 0xffff880035b35000 0t0 42713601 socket java 24739 cyclone 1010u unix 0xffff880035b35000 0t0 42713601 socket java 24739 cyclone 1011u unix 0xffff880035b35000 0t0 42713601 socket java 24739 cyclone 1012u unix 0xffff880035b35000 0t0 42713601 socket java 24739 cyclone 1013u unix 0xffff880035b35000 0t0 42713601 socket java 24739 cyclone 1014u unix 0xffff880035b35000 0t0 42713601 socket java 24739 cyclone 1015u unix 0xffff880035b35000 0t0 42713601 socket java 24739 cyclone 1016u unix 0xffff880035b35000 0t0 42713601 socket java 24739 cyclone 1017u unix 0xffff880035b35000 0t0 42713601 socket java 24739 cyclone 1018u unix 0xffff880035b35000 0t0 42713601 socket java 24739 cyclone 1019u unix 0xffff880035b35000 0t0 42713601 socket java 24739 cyclone 1020u unix 0xffff880035b35000 0t0 42713601 socket java 24739 cyclone 1021u unix 0xffff880035b35000 0t0 42713601 socket java 24739 cyclone 1022u unix 0xffff880035b35000 0t0 42713601 socket java 24739 cyclone 1023u unix 0xffff880035b35000 0t0 42713601 socket java 24739 cyclone 1024u unix 0xffff880035b35000 0t0 42713601 socket java 24739 cyclone 1025u unix 0xffff880035b35000 0t0 42713601 socket java 24739 cyclone 1026u unix 0xffff880035b35000 0t0 42713601 socket java 24739 cyclone 1027u unix 0xffff880035b35000 0t0 42713601 socket java 24739 cyclone 1028u unix 0xffff880035b35000 0t0 42713601 socket java 24739 cyclone 1029u unix 0xffff880035b35000 0t0 42713601 socket java 24739 cyclone 1030u unix 0xffff880035b35000 0t0 42713601 socket java 24739 cyclone 1031u unix 0xffff880035b35000 0t0 42713601 socket java 24739 cyclone 1032u unix 0xffff880035b35000 0t0 42713601 socket java 24739 cyclone 1033u unix 0xffff880035b35000 0t0 42713601 socket java 24739 cyclone 1034u unix 0xffff880035b35000 0t0 42713601 socket java 24739 cyclone 1035u unix 0xffff880035b35000 0t0 42713601 socket java 24739 cyclone 1036u unix 0xffff880035b35000 0t0 42713601 socket java 24739 cyclone 1037u unix 0xffff880035b35000 0t0 42713601 socket java 24739 cyclone 1038u unix 0xffff880035b35000 0t0 42713601 socket java 24739 cyclone 1039u unix 0xffff880035b35000 0t0 42713601 socket java 24739 cyclone 1040u unix 0xffff880035b35000 0t0 42713601 socket java 24739 cyclone 1041u unix 0xffff880035b35000 0t0 42713601 socket java 24739 cyclone 1042u unix 0xffff880035b35000 0t0 42713601 socket java 24739 cyclone 1043u unix 0xffff880035b35000 0t0 42713601 socket java 24739 cyclone 1044u unix 0xffff880035b35000 0t0 42713601 socket java 24739 cyclone 1045u unix 0xffff880035b35000 0t0 42713601 socket java 24739 cyclone 1046u unix 0xffff880035b35000 0t0 42713601 socket java 24739 cyclone 1047u unix 0xffff880035b35000 0t0 42713601 socket java 24739 cyclone 1048u unix 0xffff880035b35000 0t0 42713601 socket java 24739 cyclone 1049u unix 0xffff880035b35000 0t0 42713601 socket java 24739 cyclone 1050u unix 0xffff880035b35000 0t0 42713601 socket java 24739 cyclone 1051u unix 0xffff880035b35000 0t0 42713601 socket java 24739 cyclone 1052u unix 0xffff880035b35000 0t0 42713601 socket java 24739 cyclone 1053u unix 0xffff880035b35000 0t0 42713601 socket java 24739 cyclone 1054u unix 0xffff880035b35000 0t0 42713601 socket java 24739 cyclone 1055u unix 0xffff880035b35000 0t0 42713601 socket java 24739 cyclone 1056u unix 0xffff880035b35000 0t0 42713601 socket java 24739 cyclone 1057u unix 0xffff880035b35000 0t0 42713601 socket java 24739 cyclone 1058u unix 0xffff880035b35000 0t0 42713601 socket java 24739 cyclone 1059u unix 0xffff880035b35000 0t0 42713601 socket java 24739 cyclone 1060u unix 0xffff880035b35000 0t0 42713601 socket java 24739 cyclone 1061u unix 0xffff880035b35000 0t0 42713601 socket java 24739 cyclone 1062u unix 0xffff880035b35000 0t0 42713601 socket java 24739 cyclone 1063u unix 0xffff880035b35000 0t0 42713601 socket java 24739 cyclone 1064u unix 0xffff880035b35000 0t0 42713601 socket java 24739 cyclone 1065u unix 0xffff880035b35000 0t0 42713601 socket java 24739 cyclone 1066u unix 0xffff880035b35000 0t0 42713601 socket java 24739 cyclone 1067u unix 0xffff880035b35000 0t0 42713601 socket java 24739 cyclone 1068u unix 0xffff880035b35000 0t0 42713601 socket java 24739 cyclone 1069u unix 0xffff880035b35000 0t0 42713601 socket java 24739 cyclone 1070u unix 0xffff880035b35000 0t0 42713601 socket java 24739 cyclone 1071u unix 0xffff880035b35000 0t0 42713601 socket java 24739 cyclone 1072u unix 0xffff880035b35000 0t0 42713601 socket java 24739 cyclone 1073u unix 0xffff880035b35000 0t0 42713601 socket java 24739 cyclone 1074u unix 0xffff880035b35000 0t0 42713601 socket java 24739 cyclone 1075u unix 0xffff880035b35000 0t0 42713601 socket java 24739 cyclone 1076u unix 0xffff880035b35000 0t0 42713601 socket java 24739 cyclone 1077u unix 0xffff880035b35000 0t0 42713601 socket java 24739 cyclone 1078u unix 0xffff880035b35000 0t0 42713601 socket java 24739 cyclone 1079u unix 0xffff880035b35000 0t0 42713601 socket java 24739 cyclone 1080u unix 0xffff880035b35000 0t0 42713601 socket java 24739 cyclone 1081u unix 0xffff880035b35000 0t0 42713601 socket java 24739 cyclone 1082u unix 0xffff880035b35000 0t0 42713601 socket java 24739 cyclone 1083u unix 0xffff880035b35000 0t0 42713601 socket java 24739 cyclone 1084u unix 0xffff880035b35000 0t0 42713601 socket java 24739 cyclone 1085u unix 0xffff880035b35000 0t0 42713601 socket java 24739 cyclone 1086u unix 0xffff880035b35000 0t0 42713601 socket java 24739 cyclone 1087u unix 0xffff880035b35000 0t0 42713601 socket java 24739 cyclone 1088u unix 0xffff880035b35000 0t0 42713601 socket java 24739 cyclone 1089u unix 0xffff880035b35000 0t0 42713601 socket java 24739 cyclone 1090u unix 0xffff880035b35000 0t0 42713601 socket java 24739 cyclone 1091u unix 0xffff880035b35000 0t0 42713601 socket java 24739 cyclone 1092u unix 0xffff880035b35000 0t0 42713601 socket java 24739 cyclone 1093u unix 0xffff880035b35000 0t0 42713601 socket java 24739 cyclone 1094u unix 0xffff880035b35000 0t0 42713601 socket java 24739 cyclone 1095u unix 0xffff880035b35000 0t0 42713601 socket java 24739 cyclone 1096u unix 0xffff880035b35000 0t0 42713601 socket java 24739 cyclone 1097u unix 0xffff880035b35000 0t0 42713601 socket java 24739 cyclone 1098u unix 0xffff880035b35000 0t0 42713601 socket java 24739 cyclone 1099u unix 0xffff880035b35000 0t0 42713601 socket java 24739 cyclone 1100u unix 0xffff880035b35000 0t0 42713601 socket java 24739 cyclone 1101u unix 0xffff880035b35000 0t0 42713601 socket java 24739 cyclone 1102u unix 0xffff880035b35000 0t0 42713601 socket java 24739 cyclone 1103u unix 0xffff880035b35000 0t0 42713601 socket java 24739 cyclone 1104u unix 0xffff880035b35000 0t0 42713601 socket java 24739 cyclone 1105u unix 0xffff880035b35000 0t0 42713601 socket java 24739 cyclone 1106u unix 0xffff880035b35000 0t0 42713601 socket java 24739 cyclone 1107u unix 0xffff880035b35000 0t0 42713601 socket java 24739 cyclone 1108u unix 0xffff880035b35000 0t0 42713601 socket java 24739 cyclone 1109u unix 0xffff880035b35000 0t0 42713601 socket java 24739 cyclone 1110u unix 0xffff880035b35000 0t0 42713601 socket java 24739 cyclone 1111u unix 0xffff880035b35000 0t0 42713601 socket java 24739 cyclone 1112u unix 0xffff880035b35000 0t0 42713601 socket java 24739 cyclone 1113u unix 0xffff880035b35000 0t0 42713601 socket java 24739 cyclone 1114u unix 0xffff880035b35000 0t0 42713601 socket java 24739 cyclone 1115u unix 0xffff880035b35000 0t0 42713601 socket java 24739 cyclone 1116u unix 0xffff880035b35000 0t0 42713601 socket java 24739 cyclone 1117u unix 0xffff880035b35000 0t0 42713601 socket java 24739 cyclone 1118u unix 0xffff880035b35000 0t0 42713601 socket java 24739 cyclone 1119u unix 0xffff880035b35000 0t0 42713601 socket java 24739 cyclone 1120u unix 0xffff880035b35000 0t0 42713601 socket java 24739 cyclone 1121u unix 0xffff880035b35000 0t0 42713601 socket java 24739 cyclone 1122u unix 0xffff880035b35000 0t0 42713601 socket java 24739 cyclone 1123u unix 0xffff880035b35000 0t0 42713601 socket java 24739 cyclone 1124u unix 0xffff880035b35000 0t0 42713601 socket java 24739 cyclone 1125u unix 0xffff880035b35000 0t0 42713601 socket java 24739 cyclone 1126u unix 0xffff880035b35000 0t0 42713601 socket java 24739 cyclone 1127u unix 0xffff880035b35000 0t0 42713601 socket java 24739 cyclone 1128u unix 0xffff880035b35000 0t0 42713601 socket java 24739 cyclone 1129u unix 0xffff880035b35000 0t0 42713601 socket java 24739 cyclone 1130u unix 0xffff880035b35000 0t0 42713601 socket java 24739 cyclone 1131u unix 0xffff880035b35000 0t0 42713601 socket java 24739 cyclone 1132u unix 0xffff880035b35000 0t0 42713601 socket java 24739 cyclone 1133u unix 0xffff880035b35000 0t0 42713601 socket java 24739 cyclone 1134u unix 0xffff880035b35000 0t0 42713601 socket java 24739 cyclone 1135u unix 0xffff880035b35000 0t0 42713601 socket java 24739 cyclone 1136u unix 0xffff880035b35000 0t0 42713601 socket java 24739 cyclone 1137u unix 0xffff880035b35000 0t0 42713601 socket java 24739 cyclone 1138u unix 0xffff880035b35000 0t0 42713601 socket java 24739 cyclone 1139u unix 0xffff880035b35000 0t0 42713601 socket java 24739 cyclone 1140u unix 0xffff880035b35000 0t0 42713601 socket java 24739 cyclone 1141u unix 0xffff880035b35000 0t0 42713601 socket java 24739 cyclone 1142u unix 0xffff880035b35000 0t0 42713601 socket java 24739 cyclone 1143u unix 0xffff880035b35000 0t0 42713601 socket java 24739 cyclone 1144u unix 0xffff880035b35000 0t0 42713601 socket java 24739 cyclone 1145u unix 0xffff880035b35000 0t0 42713601 socket java 24739 cyclone 1146u unix 0xffff880035b35000 0t0 42713601 socket java 24739 cyclone 1147u unix 0xffff880035b35000 0t0 42713601 socket java 24739 cyclone 1148u unix 0xffff880035b35000 0t0 42713601 socket java 24739 cyclone 1149u unix 0xffff880035b35000 0t0 42713601 socket java 24739 cyclone 1150u unix 0xffff880035b35000 0t0 42713601 socket java 24739 cyclone 1151u unix 0xffff880035b35000 0t0 42713601 socket java 24739 cyclone 1152u unix 0xffff880035b35000 0t0 42713601 socket java 24739 cyclone 1153u unix 0xffff880035b35000 0t0 42713601 socket java 24739 cyclone 1154u unix 0xffff880035b35000 0t0 42713601 socket java 24739 cyclone 1155u unix 0xffff880035b35000 0t0 42713601 socket java 24739 cyclone 1156u unix 0xffff880035b35000 0t0 42713601 socket java 24739 cyclone 1157u unix 0xffff880035b35000 0t0 42713601 socket java 24739 cyclone 1158u unix 0xffff880035b35000 0t0 42713601 socket java 24739 cyclone 1159u unix 0xffff880035b35000 0t0 42713601 socket java 24739 cyclone 1160u unix 0xffff880035b35000 0t0 42713601 socket java 24739 cyclone 1161u unix 0xffff880035b35000 0t0 42713601 socket java 24739 cyclone 1162u unix 0xffff880035b35000 0t0 42713601 socket java 24739 cyclone 1163u unix 0xffff880035b35000 0t0 42713601 socket java 24739 cyclone 1164u unix 0xffff880035b35000 0t0 42713601 socket java 24739 cyclone 1165u unix 0xffff880035b35000 0t0 42713601 socket java 24739 cyclone 1166u unix 0xffff880035b35000 0t0 42713601 socket java 24739 cyclone 1167u unix 0xffff880035b35000 0t0 42713601 socket java 24739 cyclone 1168u unix 0xffff880035b35000 0t0 42713601 socket java 24739 cyclone 1169u unix 0xffff880035b35000 0t0 42713601 socket java 24739 cyclone 1170u unix 0xffff880035b35000 0t0 42713601 socket java 24739 cyclone 1171u unix 0xffff880035b35000 0t0 42713601 socket java 24739 cyclone 1172u unix 0xffff880035b35000 0t0 42713601 socket java 24739 cyclone 1173u unix 0xffff880035b35000 0t0 42713601 socket java 24739 cyclone 1174u unix 0xffff880035b35000 0t0 42713601 socket java 24739 cyclone 1175u unix 0xffff880035b35000 0t0 42713601 socket java 24739 cyclone 1176u unix 0xffff880035b35000 0t0 42713601 socket java 24739 cyclone 1177u unix 0xffff880035b35000 0t0 42713601 socket java 24739 cyclone 1178u unix 0xffff880035b35000 0t0 42713601 socket java 24739 cyclone 1179u unix 0xffff880035b35000 0t0 42713601 socket java 24739 cyclone 1180u unix 0xffff880035b35000 0t0 42713601 socket java 24739 cyclone 1181u unix 0xffff880035b35000 0t0 42713601 socket java 24739 cyclone 1182u unix 0xffff880035b35000 0t0 42713601 socket java 24739 cyclone 1183u unix 0xffff880035b35000 0t0 42713601 socket java 24739 cyclone 1184u unix 0xffff880035b35000 0t0 42713601 socket java 24739 cyclone 1185u unix 0xffff880035b35000 0t0 42713601 socket java 24739 cyclone 1186u unix 0xffff880035b35000 0t0 42713601 socket java 24739 cyclone 1187u unix 0xffff880035b35000 0t0 42713601 socket java 24739 cyclone 1188u unix 0xffff880035b35000 0t0 42713601 socket java 24739 cyclone 1189u unix 0xffff880035b35000 0t0 42713601 socket java 24739 cyclone 1190u unix 0xffff880035b35000 0t0 42713601 socket java 24739 cyclone 1191u unix 0xffff880035b35000 0t0 42713601 socket java 24739 cyclone 1192u unix 0xffff880035b35000 0t0 42713601 socket java 24739 cyclone 1193u unix 0xffff880035b35000 0t0 42713601 socket java 24739 cyclone 1194u unix 0xffff880035b35000 0t0 42713601 socket java 24739 cyclone 1195u unix 0xffff880035b35000 0t0 42713601 socket java 24739 cyclone 1196u unix 0xffff880035b35000 0t0 42713601 socket java 24739 cyclone 1197u unix 0xffff880035b35000 0t0 42713601 socket java 24739 cyclone 1198u unix 0xffff880035b35000 0t0 42713601 socket java 24739 cyclone 1199u unix 0xffff880035b35000 0t0 42713601 socket java 24739 cyclone 1200u unix 0xffff880035b35000 0t0 42713601 socket java 24739 cyclone 1201u unix 0xffff880035b35000 0t0 42713601 socket java 24739 cyclone 1202u unix 0xffff880035b35000 0t0 42713601 socket java 24739 cyclone 1203u unix 0xffff880035b35000 0t0 42713601 socket java 24739 cyclone 1204u unix 0xffff880035b35000 0t0 42713601 socket java 24739 cyclone 1205u unix 0xffff880035b35000 0t0 42713601 socket java 24739 cyclone 1206u unix 0xffff880035b35000 0t0 42713601 socket java 24739 cyclone 1207u unix 0xffff880035b35000 0t0 42713601 socket java 24739 cyclone 1208u unix 0xffff880035b35000 0t0 42713601 socket java 24739 cyclone 1209u unix 0xffff880035b35000 0t0 42713601 socket java 24739 cyclone 1210u unix 0xffff880035b35000 0t0 42713601 socket java 24739 cyclone 1211u unix 0xffff880035b35000 0t0 42713601 socket java 24739 cyclone 1212u unix 0xffff880035b35000 0t0 42713601 socket java 24739 cyclone 1213u unix 0xffff880035b35000 0t0 42713601 socket java 24739 cyclone 1214u unix 0xffff880035b35000 0t0 42713601 socket java 24739 cyclone 1215u unix 0xffff880035b35000 0t0 42713601 socket java 24739 cyclone 1216u unix 0xffff880035b35000 0t0 42713601 socket java 24739 cyclone 1217u unix 0xffff880035b35000 0t0 42713601 socket java 24739 cyclone 1218u unix 0xffff880035b35000 0t0 42713601 socket > Kafka Replica Fetcher Thread- Resource Leak > ------------------------------------------- > > Key: KAFKA-5007 > URL: https://issues.apache.org/jira/browse/KAFKA-5007 > Project: Kafka > Issue Type: Bug > Components: core, network > Affects Versions: 0.10.1.1 > Environment: Centos 7 > Jave 8 > Reporter: Joseph Aliase > Labels: reliability > > Kafka is running out of open file descriptor when system network interface is > done. > Issue description: > We have a Kafka Cluster of 5 node running on version 0.10.1.1. The open file > descriptor for the account running Kafka is set to 100000. > During an upgrade, network interface went down. Outage continued for 12 hours > eventually all the broker crashed with java.io.IOException: Too many open > files error. > We repeated the test in a lower environment and observed that Open Socket > count keeps on increasing while the NIC is down. > We have around 13 topics with max partition size of 120 and number of replica > fetcher thread is set to 8. > Using an internal monitoring tool we observed that Open Socket descriptor > for the broker pid continued to increase although NIC was down leading to > Open File descriptor error. -- This message was sent by Atlassian JIRA (v6.3.15#6346)