[Box Backup-dev] COMMIT r476 - box/chris/general/bin/bbackupd

[Chris Wilson]

Hi Charles,

On Mon, 2006-02-20 at 12:16 +0000, Charles Lecklider wrote:
> Ben Summers wrote:
> > Maybe I'm wrong in thinking that you can wait on pipes.
> 
> You can. If you want to wait using the handle as the event you need to
> be careful you don't try to read and write at the same time (since you
> won't be able to tell which happened), but if you create an event for
> the OVERLAPPED you're fine.

In my tests, the problem I had was that the pipe handle remained
signaled after I had read from it, and I couldn't see how to reset it.
So we would end up thinking that we had more data to read, and call
ReadFile, which blocks... same problem.

At least with an Event the behaviour is well defined and controllable.

> Something like this is certainly necessary, but I think this is the
> wrong design. Having an extra thread just to talk to the command socket
> is incredibly wasteful. With overlapped IO you can do all the IO on a
> single thread with no blocking - but not like this.

Not my design, I inherited it. As far as I can tell, there is a need for
a separate thread - a control client can connect at any time, even when
the main thread is busy doing a backup, and request it to terminate or
sync again. Without threads I don't know how that would work. 

I don't mind throwing away the separate thread that I inherited, or this
new patch which merely fixes the most obvious bugs, but I want to be
sure that we don't lose any functionality by doing so. 

Why is an extra thread wasteful? I know that synchronisation would be
easier without it, and multithreaded programming is always tricky, but I
thought that threads were lightweight, and this one (now) spends almost
all its time in WaitForMultipleObjects.

> Chris: If you're intent on learning overlapped IO you want to be looking
> at completion ports. They're essentially a message queue, but without
> all that tedious messing around with critical sections etc. You can wait
> on them from multiple threads, and the wait has a timeout (which is what
> I was planning on using for SSL keepalives - if no completion packet
> comes in after X ms send the keepalive - no timer required).

OK, thanks, I will have a look at them. What API functions are involved?

> However, once you start looking at it you'll soon realise that there's a
> lot more that needs to change in order to take advantage of completion
> ports properly. The good news is that none of the actual box code needs
> to change (or at least I didn't see any in my quick review), and the new
> code will be simpler in a lot of ways than the old, but the bad news is
> that pretty much the entire Win32 architecture needs to change.
> 
> That's why I wasn't planning on working on it until 0.10 is done - start
> adding this now and there'll never be a release....

I'm not sure if it will go into 0.10, but this BADLY needs fixing. The
command socket is so dangerous as to be almost unusable on win32.

Cheers, Chris.
-- 
  ___ __     _
 / __/ / ,__(_)_  | Chris Wilson <0000 at qwirx.com> - Cambs UK |
/ (_/ ,\/ _/ /_ \ | Security/C/C++/Java/Perl/SQL/HTML Developer |
\ _/_/_/_//_/___/ | We are GNU-free your mind-and your software |