Re: Help please！！！ snort_build261 can not reload config successfully with daq in nfq

["Michael Altizer \(mialtize\) via Snort-users" <snort-users () lists snort org>]

If I had to guess, I'd say that nl_socket_recv() probably properly set EINTR while I was originally developing it and 
something has changed.  Anyway, having the check in the main loop is appropriate and matches the behavior of the other 
DAQ modules.  I've added it now; please try with the latest master branch from libdaq.  Thanks for reporting.

On 10/10/19 5:34 AM, sofardware via Snort-users wrote:
Hi,
      After my debuging , I found a direct but not basic reason:
      In the function  nfq_daq_msg_receive in  libdaq-master\modules\nfq\daq_nfq.c,  the interrupt processing need  
nl_socket_recv() return EINTR.  But  there was no EINTR returned when reload interrupt happen.
     But in  pacap_daq_msg_receive and afpacket_daq_msg_receive,  the interrupt processing  is putted in while() 
directly.

     Now I  add   the interrupt processing   in while() directly like that in pcap_daq_msg_receive，then  reload_config 
command works successfully.
     But  can you tell me  if  the above way for  resolving the problem   can lead to other problems???   and 
secondly<https://www.baidu.com/link?url=DLH1s96E9ae2ibiWplYpGeimSy4Mjky2eBhUOAX-jfuMSJnKHOIj9pjUFrBWT5lZ__tCmlxuKMrYrQq6sTaMxSdINktZq_9a_s91SpmXV2O&wd=&eqid=f75c30f7000f550c000000025d9ef3f0>,
  why the original processing  need  nl_socket_recv() return EINTR，and why it can not come out in fact？？


=================
static unsigned nfq_daq_msg_receive(void *handle, const unsigned max_recv, const DAQ_Msg_t *msgs[], DAQ_RecvStatus 
*rstat)
{
    Nfq_Context_t *nfqc = (Nfq_Context_t *) handle;
    unsigned idx = 0;
    *rstat = DAQ_RSTAT_OK;
    while (idx < max_recv)
    {
        /* Make sure that we have a packet descriptor available to populate. */
        NfqPktDesc *desc = nfqc->pool.freelist;
        if (!desc)
        {
            *rstat = DAQ_RSTAT_NOBUF;
            break;
        }

                  /* added by me for reload fail*/
                  if (nfqc->interrupted)
                 {
                        nfqc->interrupted = false;
                        *rstat = DAQ_RSTAT_INTERRUPTED;
                          break;
                  }
                /* added by me for reload fail*/
        ssize_t ret = nl_socket_recv(nfqc, desc->nlmsg_buf, nfqc->nlmsg_bufsize, idx == 0);
        if (ret < 0)
        {
            if (errno == ENOBUFS)
            {
                nfqc->stats.hw_packets_dropped++;
                continue;
            }
            else if (errno == EAGAIN || errno == EWOULDBLOCK)
                *rstat = (idx == 0) ? DAQ_RSTAT_TIMEOUT : DAQ_RSTAT_WOULD_BLOCK;
            else if (errno == EINTR)    // the original processing
            {
                if (!nfqc->interrupted)
                    continue;
                nfqc->interrupted = false;
                *rstat = DAQ_RSTAT_INTERRUPTED;
            }
======================================================================================================================================================
static unsigned nfq_daq_msg_receive(void *handle, const unsigned max_recv, const DAQ_Msg_t *msgs[], DAQ_RecvStatus 
*rstat)
{
    Nfq_Context_t *nfqc = (Nfq_Context_t *) handle;
    unsigned idx = 0;
    *rstat = DAQ_RSTAT_OK;
    while (idx < max_recv)
    {
        /* Make sure that we have a packet descriptor available to populate. */
        NfqPktDesc *desc = nfqc->pool.freelist;
        if (!desc)
        {
            *rstat = DAQ_RSTAT_NOBUF;
            break;
        }
        ssize_t ret = nl_socket_recv(nfqc, desc->nlmsg_buf, nfqc->nlmsg_bufsize, idx == 0);
        if (ret < 0)
        {
            if (errno == ENOBUFS)
            {
                nfqc->stats.hw_packets_dropped++;
                continue;
            }
            else if (errno == EAGAIN || errno == EWOULDBLOCK)
                *rstat = (idx == 0) ? DAQ_RSTAT_TIMEOUT : DAQ_RSTAT_WOULD_BLOCK;
            else if (errno == EINTR)
            {
                if (!nfqc->interrupted)
                    continue;
                nfqc->interrupted = false;
                *rstat = DAQ_RSTAT_INTERRUPTED;
            }
            else
            {
                SET_ERROR(nfqc->modinst, "%s: Socket receive failed: %zd - %s (%d)",
                        __func__, ret, strerror(errno), errno);
                *rstat = DAQ_RSTAT_ERROR;
            }
            break;
        }
        errno = 0;
        ret = mnl_cb_run(desc->nlmsg_buf, ret, 0, nfqc->portid, process_message_cb, desc);
        if (ret < 0)
        {
            SET_ERROR(nfqc->modinst, "%s: Netlink message processing failed: %zd - %s (%d)",
                    __func__, ret, strerror(errno), errno);
            *rstat = DAQ_RSTAT_ERROR;
            break;
        }
        /* Increment the module instance's packet counter. */
        nfqc->stats.packets_received++;
        /* Last, but not least, extract this descriptor from the free list and
            place the message in the return vector. */
        nfqc->pool.freelist = desc->next;
        desc->next = NULL;
        nfqc->pool.info.available--;
        msgs[idx] = &desc->msg;
        idx++;
    }
==========================================================================
static unsigned pcap_daq_msg_receive(void *handle, const unsigned max_recv, const DAQ_Msg_t *msgs[], DAQ_RecvStatus 
*rstat)
{
    struct pcap_pkthdr *pcaphdr;
    Pcap_Context_t *pc = (Pcap_Context_t *) handle;
    const u_char *data;
    unsigned idx;
    *rstat = DAQ_RSTAT_OK;
    for (idx = 0; idx < max_recv; idx++)
    {
        /* Check to see if the receive has been canceled.  If so, reset it and return appropriately. */
        if (pc->interrupted)
        {
            pc->interrupted = false;
            *rstat = DAQ_RSTAT_INTERRUPTED;
            break;
        }
        /* If there is a pending descriptor from the readback timeout feature, check if it's ready
            to be realized.  If it is, finish receiving it and carry on. */
        if (pc->pending_desc)
        {
            struct timeval delta;
            timersub(&pc->pending_desc->pkthdr.ts, &pc->last_recv, &delta);
            if (timercmp(&delta, &pc->timeout_tv, >))
            {
                timeradd(&pc->last_recv, &pc->timeout_tv, &pc->last_recv);
                *rstat = DAQ_RSTAT_TIMEOUT;
                break;
            }
            pc->last_recv = pc->pending_desc->pkthdr.ts;
            pc->pool.info.available--;
            msgs[idx] = &pc->pending_desc->msg;
            pc->stats.packets_received++;
            pc->pending_desc = NULL;
            continue;
        }
        /* Make sure that we have a packet descriptor available to populate *before*
            calling into libpcap. */
        PcapPktDesc *desc = pc->pool.freelist;
        if (!desc)
        {
            *rstat = DAQ_RSTAT_NOBUF;
            break;
        }
        /* When dealing with a live interface, try to get the first packet in non-blocking mode.
            If there's nothing to receive, switch to blocking mode. */
        int pcap_rval;
        if (pc->mode != DAQ_MODE_READ_FILE && idx == 0)
        {
            if (set_nonblocking(pc, true) != DAQ_SUCCESS)
            {
                *rstat = DAQ_RSTAT_ERROR;
                break;
            }
            pcap_rval = pcap_next_ex(pc->handle, &pcaphdr, &data);
            if (pcap_rval == 0)
            {
                if (set_nonblocking(pc, false) != DAQ_SUCCESS)
                {
                    *rstat = DAQ_RSTAT_ERROR;
                    break;
                }
                pcap_rval = pcap_next_ex(pc->handle, &pcaphdr, &data);
            }
        }
        else
            pcap_rval = pcap_next_ex(pc->handle, &pcaphdr, &data);
        if (pcap_rval <= 0)
        {
            if (pcap_rval == 0)
                *rstat = (idx == 0) ? DAQ_RSTAT_TIMEOUT : DAQ_RSTAT_WOULD_BLOCK;
            else if (pcap_rval == -1)
            {
                SET_ERROR(pc->modinst, "%s", pcap_geterr(pc->handle));
                *rstat = DAQ_RSTAT_ERROR;
            }
            else if (pcap_rval == -2)
            {
                /* LibPCAP brilliantly decides to return -2 if it hit EOF in readback OR pcap_breakloop()
                    was called.  Let's try to differentiate by checking to see if we asked for a break. */
                if (!pc->interrupted && pc->mode == DAQ_MODE_READ_FILE)
                {
                    /* Insert a final timeout receive status when readback timeout mode is enabled. */
                    if (pc->readback_timeout && !pc->final_readback_timeout)
                    {
                        pc->final_readback_timeout = true;
                        *rstat = DAQ_RSTAT_TIMEOUT;
                    }
                    else
                        *rstat = DAQ_RSTAT_EOF;
                }
                else
                {
                    pc->interrupted = false;
                    *rstat = DAQ_RSTAT_INTERRUPTED;
                }
            }
            break;
        }
        /* Update hw packet counters to make sure we detect counter overflow */
        if (++pc->hwupdate_count == DAQ_PCAP_ROLLOVER_LIM)
            update_hw_stats(pc);
        /* Populate the packet descriptor */
        int caplen = (pcaphdr->caplen > pc->snaplen) ? pc->snaplen : pcaphdr->caplen;
        memcpy(desc->data, data, caplen);
        /* Next, set up the DAQ message.  Most fields are prepopulated and unchanging. */
        DAQ_Msg_t *msg = &desc->msg;
        msg->data_len = caplen;
        /* Then, set up the DAQ packet header. */
        DAQ_PktHdr_t *pkthdr = &desc->pkthdr;
        pkthdr->pktlen = pcaphdr->len;
        pkthdr->ts.tv_sec = pcaphdr->ts.tv_sec;
        pkthdr->ts.tv_usec = pcaphdr->ts.tv_usec;
        /* Last, but not least, extract this descriptor from the free list and
            place the message in the return vector. */
        pc->pool.freelist = desc->next;
        desc->next = NULL;
        /* If the readback timeout feature is enabled, check to see if the configured timeout has
            elapsed between the previous packet and this one.  If it has, store the descriptor for
            later without modifying counters and return the timeout receive status. */
        if (pc->mode == DAQ_MODE_READ_FILE && pc->readback_timeout && pc->timeout > 0)
        {
            if (timerisset(&pc->last_recv) && timercmp(&pkthdr->ts, &pc->last_recv, >))
            {
                struct timeval delta;
                timersub(&pkthdr->ts, &pc->last_recv, &delta);
                if (timercmp(&delta, &pc->timeout_tv, >))
                {
                    pc->pending_desc = desc;
                    timeradd(&pc->last_recv, &pc->timeout_tv, &pc->last_recv);
                    *rstat = DAQ_RSTAT_TIMEOUT;
                    break;
                }
            }
            pc->last_recv = pkthdr->ts;
        }
        pc->pool.info.available--;
        msgs[idx] = &desc->msg;
        /* Finally, increment the module instance's packet counter. */
        pc->stats.packets_received++;
    }
    return idx;
}






At 2019-10-10 10:37:54, "Russ Combs (rucombs)" <rucombs () cisco com><mailto:rucombs () cisco com> wrote:
Does Ctrl+C exit normally with the NFQ DAQ without reload?

From: sofardware <sofardware () 126 com><mailto:sofardware () 126 com>
Date: Wednesday, October 9, 2019 at 10:13 PM
To: "Tom Peters (thopeter)" <thopeter () cisco com><mailto:thopeter () cisco com>
Cc: "Shravan Rangarajuvenkata (shrarang)" <shrarang () cisco com><mailto:shrarang () cisco com>, "Snort-users () lists 
snort org"<mailto:Snort-users () lists snort org> <Snort-users () lists snort org><mailto:Snort-users () lists snort 
org>, "Russ Combs (rucombs)" <rucombs () cisco com><mailto:rucombs () cisco com>
Subject: Help please！！！ snort_build261 can not reload config successfully with daq in nfq


Hi,
      I am   anxious to  resolve this problem. Please give me some help. Thank you very much.
      I have read  README file in snort3 and DAQ, and did not found useful info for this problem.
-----------------------
Hi，
   I need help for this：
   snort_build261 can not reload config successfully  with daq in nfq， and also can not  be exit by  pressing keys 
“Ctrl+C”.
   But it works well  with daq of not nfq.

[root@localhost build]# /usr/local/snort261/bin/snort --daq-dir /usr/local/lib/daq/ --daq nfq -i 1 -c 
/usr/local/snort261/etc/snort/snort.lua --shell -j
--------------------------------------------------
o")~   Snort++ 3.0.0-261
--------------------------------------------------
Loading /usr/local/snort261/etc/snort/snort.lua:
Loading snort_defaults.lua:
Finished snort_defaults.lua:
Loading file_magic.lua:
Finished file_magic.lua:
 ssh
 host_cache
 pop
 binder
 stream_tcp
 network
 gtp_inspect
 packets
 dce_http_proxy
 stream_icmp
 normalizer
 ftp_server
 stream_udp
 search_engine
 ips
 dce_smb
 latency
 wizard
 appid
 file_id
 ftp_data
 hosts
 smtp
 port_scan
 dce_http_server
 modbus
 dce_tcp
 telnet
 host_tracker
 ssl
 sip
 rpc_decode
 http2_inspect
 http_inspect
 back_orifice
 stream_user
 stream_ip
 classifications
 dnp3
 active
 ftp_client
 daq
 decode
 alerts
 stream
 references
 arp_spoof
 output
 dns
 dce_udp
 imap
 process
 stream_file
Finished /usr/local/snort261/etc/snort/snort.lua:
--------------------------------------------------
/usr/local/lib/daq//daq_afpacket.so: Module API version (0x10007) differs from expected version (0x30001)
/usr/local/lib/daq//daq_afpacket.so: Failed to register DAQ module.
/usr/local/lib/daq//daq_ipfw.so: Module API version (0x10007) differs from expected version (0x30001)
/usr/local/lib/daq//daq_ipfw.so: Failed to register DAQ module.
nfq DAQ configured to passive.
Commencing packet processing
Entering command shell
o")~
++ [0] 1
reload_config('/usr/local/snort261/etc/snort/snort.lua')
.. reloading configuration
Loading /usr/local/snort261/etc/snort/snort.lua:
Loading snort_defaults.lua:
Finished snort_defaults.lua:
Loading file_magic.lua:
Finished file_magic.lua:
 ssh
 host_cache
 pop
 binder
 stream_tcp
 network
 gtp_inspect
 packets
 dce_http_proxy
 stream_icmp
 normalizer
 ftp_server
 stream_udp
 search_engine
 ips
 dce_smb
 latency
 wizard
 appid
 file_id
 ftp_data
 hosts
 smtp
 port_scan
 dce_http_server
 modbus
 dce_tcp
 telnet
 host_tracker
 ssl
 sip
 rpc_decode
 http2_inspect
 http_inspect
 back_orifice
 stream_user
 stream_ip
 classifications
 dnp3
 active
 ftp_client
 daq
 decode
 alerts
 stream
 references
 arp_spoof
 output
 dns
 dce_udp
 imap
 process
 stream_file
Finished /usr/local/snort261/etc/snort/snort.lua:
0 hosts loaded
reload_config('/usr/local/snort261/etc/snort/snort.lua')
== reload pending; retry
^C** caught int signal
== stopping
^C** caught int signal
== stopping
^C** caught int signal
== stopping
^C** caught int signal
== stopping


==============================================================================no nfq================
[root@localhost build]# /usr/local/snort261/bin/snort --daq-dir /usr/local/lib/daq/ -i eth0 -c 
/usr/local/snort261/etc/snort/snort.lua --shell -j
--------------------------------------------------
o")~   Snort++ 3.0.0-261
--------------------------------------------------
Loading /usr/local/snort261/etc/snort/snort.lua:
Loading snort_defaults.lua:
Finished snort_defaults.lua:
Loading file_magic.lua:
Finished file_magic.lua:
 ssh
 host_cache
 pop
 binder
 stream_tcp
 network
 gtp_inspect
 packets
 dce_http_proxy
 stream_icmp
 normalizer
 ftp_server
 stream_udp
 search_engine
 ips
 dce_smb
 latency
 wizard
 appid
 file_id
 ftp_data
 hosts
 smtp
 port_scan
 dce_http_server
 modbus
 dce_tcp
 telnet
 host_tracker
 ssl
 sip
 rpc_decode
 http2_inspect
 http_inspect
 back_orifice
 stream_user
 stream_ip
 classifications
 dnp3
 active
 ftp_client
 daq
 decode
 alerts
 stream
 references
 arp_spoof
 output
 dns
 dce_udp
 imap
 process
 stream_file
Finished /usr/local/snort261/etc/snort/snort.lua:
--------------------------------------------------
/usr/local/lib/daq//daq_afpacket.so: Module API version (0x10007) differs from expected version (0x30001)
/usr/local/lib/daq//daq_afpacket.so: Failed to register DAQ module.
/usr/local/lib/daq//daq_ipfw.so: Module API version (0x10007) differs from expected version (0x30001)
/usr/local/lib/daq//daq_ipfw.so: Failed to register DAQ module.
pcap DAQ configured to passive.
Commencing packet processing
Entering command shell
o")~
++ [0] eth0
reload_config('/usr/local/snort261/etc/snort/snort.lua')
.. reloading configuration
Loading /usr/local/snort261/etc/snort/snort.lua:
Loading snort_defaults.lua:
Finished snort_defaults.lua:
Loading file_magic.lua:
Finished file_magic.lua:
 ssh
 host_cache
 pop
 binder
 stream_tcp
 network
 gtp_inspect
 packets
 dce_http_proxy
 stream_icmp
 normalizer
 ftp_server
 stream_udp
 search_engine
 ips
 dce_smb
 latency
 wizard
 appid
 file_id
 ftp_data
 hosts
 smtp
 port_scan
 dce_http_server
 modbus
 dce_tcp
 telnet
 host_tracker
 ssl
 sip
 rpc_decode
 http2_inspect
 http_inspect
 back_orifice
 stream_user
 stream_ip
 classifications
 dnp3
 active
 ftp_client
 daq
 decode
 alerts
 stream
 references
 arp_spoof
 output
 dns
 dce_udp
 imap
 process
 stream_file
Finished /usr/local/snort261/etc/snort/snort.lua:
0 hosts loaded
.. swapping configuration
== reload complete
o")~
























_______________________________________________
Snort-users mailing list
Snort-users () lists snort org<mailto:Snort-users () lists snort org>
Go to this URL to change user options or unsubscribe:
https://lists.snort.org/mailman/listinfo/snort-users

        To unsubscribe, send an email to:
        snort-users-leave () lists snort org<mailto:snort-users-leave () lists snort org>

Please visit http://blog.snort.org to stay current on all the latest Snort news!

Please follow these rules: https://snort.org/faq/what-is-the-mailing-list-etiquette


_______________________________________________
Snort-users mailing list
Snort-users () lists snort org
Go to this URL to change user options or unsubscribe:
https://lists.snort.org/mailman/listinfo/snort-users

        To unsubscribe, send an email to:
        snort-users-leave () lists snort org

Please visit http://blog.snort.org to stay current on all the latest Snort news!

Please follow these rules: https://snort.org/faq/what-is-the-mailing-list-etiquette