Ticket #40 (new defect)

Opened 4 months ago

Last modified 3 months ago

making several calls to send_data with small chunks of data is significantly slower than making a few calls with bigger chunks

Reported by: tmm1 Assigned to: tmm1
Priority: minor Milestone: EventMachine "Framework"
Keywords: Cc: francis wyhaines

Description 

wyhaines: I have noticed that making several calls to send_data with small chunks of data is significantly slower than making a few calls with bigger chunks.
aman: ooh interesting
aman: any idea why
wyhaines: Yeah.  So, when I send static files, I will aggregate strings together into one bigger string and accept the cost for that instead of the cost for several small sends.
wyhaines: I had started looking at the code that handles the send queue.  It's _supposed_ to aggregate small chunks itself, but I think there's just something not working right in there.
wyhaines: I haven't looked at it enough to figure it out, though.
wyhaines: I do think there is a problem there, though.
wyhaines: The C++ code should be able to aggregate chunks more efficiently than doing it on the Ruby side.

Attachments

1741.diff (2.4 kB) - added by tmm1 on 06/30/08 18:24:26.
writev instead of send

Change History

06/30/08 18:24:05 changed by tmm1

One possible approach is to let the kernel handle the coalescing of multiple OutboundData? pages instead of doing it in the reactor: http://p.ramaze.net/1741. However, in my tests, this did not make a significant difference.

06/30/08 18:24:26 changed by tmm1

  • attachment 1741.diff added.

writev instead of send

06/30/08 19:56:19 changed by raggi

Awesome, I will poke Francis on this when I next speak to him.

This may actually solve another ticket.

07/09/08 20:32:43 changed by tmm1

fix memory leak in my patch: 

--- a/ext/ed.cpp
+++ b/ext/ed.cpp
@@ -627,11 +627,12 @@ void ConnectionDescriptor::_WriteOutboundData()
 
                int sent = bytes_written;
                for(int i = 0; i < iovcnt; i++) {
+                       OutboundPage *op = &(OutboundPages[0]);
                        if (iov[i].iov_len <= sent) {
+                               op->Free();
                                OutboundPages.pop_front();
                                sent -= iov[i].iov_len;
                        } else {
-                               OutboundPage *op = &(OutboundPages[i]);
                                op->Offset += sent;
                        }
                }

07/22/08 12:49:25 changed by raggi

  • priority changed from critical to minor.

demoted to minor until it comes out of research dev

07/22/08 15:19:35 changed by raggi

  • owner changed from raggi to tmm1.

Re-assign to tmm1 for tracking, patch can be applied by whoever when ready.