From trac at boxbackup.org Sat Nov 3 12:00:01 2012 From: trac at boxbackup.org (trac at boxbackup.org) Date: Sat, 3 Nov 2012 12:00:01 +0000 (GMT) Subject: [Boxbackup-dev] Current open tickets Message-ID: <20121103120001.82404198C3D@www.boxbackup.org> Note: to view an indiviual ticket, use: https://www.boxbackup.org/trac/ticket/(number) The following is a listing of current problems submitted by Box Backup users. These represent problem reports covering all versions including experimental development code and obsolete releases. S Ticket Owner Component Summary - ------ ------ ------------- ------------------------------------------------------------ n 4 martin box libraries Port Box Backup to AIX n 6 box libraries Contribute code: SMTP client, HTTP server, Database drivers, n 7 box libraries Improve restore speed on local repositories n 8 chris box libraries Improve handling of directories with many files n 13 chris bbackupd Fix file locking on Windows n 14 chris bbackupd Fix large file issues on Windows n 16 chris bbackupquery Restore deleted directories may fail a 17 chris bbackupquery List files using wildcards a 20 chris bbackupctl bbackupctl reload reports prior settings n 45 ben bbackupd File diff performance patch (reduced disk IO and wall time n 46 chris bbackupd bbackupd only ever saves reverse diffs, corrupted files on s n 47 chris bbackupd Account numbers greater than 2^31 (0x7fffffff) do not work c n 48 chris bbackupd Locations that don't exist on first run are never tried agai n 49 chris bbackupd ID map (rename tracking) broken since [288] n 50 chris bbackupquery No way to capture stderr under Windows n 51 chris bbackupd No way to force bbackupd to re-upload files under Windows n 52 chris bbackupd Unable to control the maintenance of old vs. deleted files n 53 chris bbackupd Comparing root directory locations does not work under Windo n 54 chris bbackupd Locations not found on disk (e.g. unmounted filesystems) can n 55 chris bbackupd Should store and preserve directory timestamps n 56 bbackupquery Windows: User can polute restore directory by adding a trail n 57 ben bbackupd Create boxbackupresume and other files in DataDirectory n 58 ben bbackupd bbackupquery connection timeout results in error message, sh n 59 ben bbackupd Inconsistent treatment of NotifyScript between Box Backup an n 60 ben bbstored Case sensitivity between bbstored-certs and Boxi n 62 ben bbackupd Backups of deleted files may be removed very quickly by hous a 63 chris bbackupd Support hard links in directories n 65 ben bbackupd Some file attributes under Windows are not backed up n 66 ben bbackupd Windows ACL support a 67 chris bbackupd Add mutexes for Win32 (client) to enable detection of runnin n 68 ben bbackupd More details when ReadError occurs n 69 bbackupquery CRTL+C closes bbackupquery console window under Windows n 70 ben bbackupd Non-ascii character cause error for notifyscript path n 71 ben bbackupd bbackupd.log gets randomly deleted n 72 test suite Patch intercept.h and intercept.c for NetBSD 4 and 5 n 74 bbackupquery Reconnect bbackupquery automatically, or at least have a "co n 75 ben bbackupd Restore times wrong, times on store correct n 77 ben bbackupd Min verbosity switch broken for bbackupquery a 78 chris bbackupd Error level for "bbackupquery.exe -u" 39 tickets total. From trac at boxbackup.org Sat Nov 10 12:00:00 2012 From: trac at boxbackup.org (trac at boxbackup.org) Date: Sat, 10 Nov 2012 12:00:00 +0000 (GMT) Subject: [Boxbackup-dev] Current open tickets Message-ID: <20121110120000.E887B198226@www.boxbackup.org> Note: to view an indiviual ticket, use: https://www.boxbackup.org/trac/ticket/(number) The following is a listing of current problems submitted by Box Backup users. These represent problem reports covering all versions including experimental development code and obsolete releases. S Ticket Owner Component Summary - ------ ------ ------------- ------------------------------------------------------------ n 4 martin box libraries Port Box Backup to AIX n 6 box libraries Contribute code: SMTP client, HTTP server, Database drivers, n 7 box libraries Improve restore speed on local repositories n 8 chris box libraries Improve handling of directories with many files n 13 chris bbackupd Fix file locking on Windows n 14 chris bbackupd Fix large file issues on Windows n 16 chris bbackupquery Restore deleted directories may fail a 17 chris bbackupquery List files using wildcards a 20 chris bbackupctl bbackupctl reload reports prior settings n 45 ben bbackupd File diff performance patch (reduced disk IO and wall time n 46 chris bbackupd bbackupd only ever saves reverse diffs, corrupted files on s n 47 chris bbackupd Account numbers greater than 2^31 (0x7fffffff) do not work c n 48 chris bbackupd Locations that don't exist on first run are never tried agai n 49 chris bbackupd ID map (rename tracking) broken since [288] n 50 chris bbackupquery No way to capture stderr under Windows n 51 chris bbackupd No way to force bbackupd to re-upload files under Windows n 52 chris bbackupd Unable to control the maintenance of old vs. deleted files n 53 chris bbackupd Comparing root directory locations does not work under Windo n 54 chris bbackupd Locations not found on disk (e.g. unmounted filesystems) can n 55 chris bbackupd Should store and preserve directory timestamps n 56 bbackupquery Windows: User can polute restore directory by adding a trail n 57 ben bbackupd Create boxbackupresume and other files in DataDirectory n 58 ben bbackupd bbackupquery connection timeout results in error message, sh n 59 ben bbackupd Inconsistent treatment of NotifyScript between Box Backup an n 60 ben bbstored Case sensitivity between bbstored-certs and Boxi n 62 ben bbackupd Backups of deleted files may be removed very quickly by hous a 63 chris bbackupd Support hard links in directories n 65 ben bbackupd Some file attributes under Windows are not backed up n 66 ben bbackupd Windows ACL support a 67 chris bbackupd Add mutexes for Win32 (client) to enable detection of runnin n 68 ben bbackupd More details when ReadError occurs n 69 bbackupquery CRTL+C closes bbackupquery console window under Windows n 70 ben bbackupd Non-ascii character cause error for notifyscript path n 71 ben bbackupd bbackupd.log gets randomly deleted n 72 test suite Patch intercept.h and intercept.c for NetBSD 4 and 5 n 74 bbackupquery Reconnect bbackupquery automatically, or at least have a "co n 75 ben bbackupd Restore times wrong, times on store correct n 77 ben bbackupd Min verbosity switch broken for bbackupquery a 78 chris bbackupd Error level for "bbackupquery.exe -u" 39 tickets total. From bb at grandville.net Sun Nov 11 13:49:11 2012 From: bb at grandville.net (bb at grandville.net) Date: Sun, 11 Nov 2012 14:49:11 +0100 Subject: [Boxbackup-dev] [common/Windows] logging to console Message-ID: <00b501cdc013$554eed80$ffecc880$@grandville.net> Hi All, Some of the error message reported to the console are system's localized messages (with accents in my case) and not UTF8 encoded as expected. I wrote a function to report correctly this kind of text stream to the console My patch is attached Arnaud -------------- next part -------------- A non-text attachment was scrubbed... Name: ConsoleLogging.patch Type: application/octet-stream Size: 1471 bytes Desc: not available URL: From chris at qwirx.com Sun Nov 11 19:23:57 2012 From: chris at qwirx.com (Chris Wilson) Date: Sun, 11 Nov 2012 19:23:57 +0000 (GMT) Subject: [Boxbackup-dev] [common/Windows] logging to console In-Reply-To: <00b501cdc013$554eed80$ffecc880$@grandville.net> References: <00b501cdc013$554eed80$ffecc880$@grandville.net> Message-ID: Hi Arnaud, On Sun, 11 Nov 2012, bb at grandville.net wrote: > Some of the error message reported to the console are system's localized > messages (with accents in my case) and not UTF8 encoded as expected. I > wrote a function to report correctly this kind of text stream to the > console My patch is attached Thanks for the patch, but I suspect this is not the correct fix. The aim of these conversions is to fix filename reporting, which should always be in UTF-8, because we convert all other encodings to UTF-8 in emu.cpp and only use UTF-8 internally. I'd like to see examples of the kinds of messages you report, and also what effect this has on log messages with filenames containing accents. I suspect that the latter would be broken by the patch. Perhaps we need to convert the sources of those messages into UTF-8 as well, e.g. GetLastError(), but I'd need to see the messages to understand whether that's the case. Cheers, Chris. -- _____ __ _ \ __/ / ,__(_)_ | Chris Wilson Cambs UK | / (_/ ,\/ _/ /_ \ | Security/C/C++/Java/Ruby/Perl/SQL Developer | \__/_/_/_//_/___/ | We are GNU : free your mind & your software | From bb at grandville.net Sun Nov 11 08:59:50 2012 From: bb at grandville.net (bb at grandville.net) Date: Sun, 11 Nov 2012 09:59:50 +0100 Subject: [Boxbackup-dev] [bbstoreaccounts] folder remove Message-ID: <009d01cdbfea$e95b4e20$bc11ea60$@grandville.net> Hi All, I'm using the stunning boxbackup on Windows platform, and I found a problem when deleting an account, C:\dev\boxbackup\Release>bbstoreaccounts -c C:\dev\boxbackup\conf\boxbackup\bbstored.conf delete 101 yes NOTICE: Deleting store directory C:\dev\boxbackup\Data\Single\backup\00000101\... ERROR: Failed to delete files in C:\dev\boxbackup\Data\Single\backup\00000101\, delete them manually. In fact, the user's folder contains an open file write.lock so before deleting his folders wouldn't be mandatory to close this file ?!?! I take this opportunity to add a notice at the end of the DeleteAccount to validate the successful completion of the call. My proposed patch (bbstoreaccounts.cpp.patch) is attached. The command to recursively delete windows folder would be rd /s /q C:\dev\boxbackup\Data\Single\backup\00000101 I wrote a rmdir function specific to windows in the second patch and I reorder emu_stat struct from large to narrow items) to reduce the data alignment. Hope this could help, Arnaud -------------- next part -------------- An HTML attachment was scrubbed... URL: -------------- next part -------------- A non-text attachment was scrubbed... Name: bbstoreaccounts.cpp.patch Type: application/octet-stream Size: 460 bytes Desc: not available URL: -------------- next part -------------- A non-text attachment was scrubbed... Name: rmdir.patch Type: application/octet-stream Size: 3362 bytes Desc: not available URL: From bb at grandville.net Sun Nov 11 21:45:50 2012 From: bb at grandville.net (bb at grandville.net) Date: Sun, 11 Nov 2012 22:45:50 +0100 Subject: [Boxbackup-dev] [common/Windows] logging to console In-Reply-To: References: <00b501cdc013$554eed80$ffecc880$@grandville.net> Message-ID: <001201cdc055$eb373770$c1a5a650$@grandville.net> Hi, I understand what you mean, but I found another problem. To validate your suspicion, I tried to use a root backup location folder name with some accents :-)). The call to the windows version of BackupDaemon::statfs at 2368 failed because Location.mPath must be UTF8 encoded but the rConfig.GetKeyValue("Path") failed when it try to read my bbackupd.conf file recorded in UTF8 format in place of the standard ASCII !?!? INFO: Opening connection to server 'localhost'... INFO: Connection made, login successful WARNING: Exception thrown: CommonException(OSFileError) (Failed to stat path 'C:\Voix ambigu?' for location 'temp': No such file or directory (2)) at c:\dev\boxbackup\bin\bbackupd\backupdaemon.cpp:2375 ERROR: Failed to configure location 'temp' path 'C:\Voix ambigu?': Common OSFileError: please check for previous errors NOTICE: 1 redundant locations in root directory found, will delete from store after 172800 seconds. INFO: Unused location in root: bsd INFO: Not deleting unused entries - too early (172800 seconds remaining) ERROR: VSS: Failed to create backup components: Acc?s refus?. (0x80070005) INFO: Saved store object info file version 2 (C:\dev\boxbackup\var\bbackupd.state) INFO: Not notifying administrator about event read-error, set NotifyScript to do this in future NOTICE: Finished scan of local files NOTICE: File statistics: total file size uploaded 0, bytes already on server 0, encoded size 0, 0 files uploaded, 0 dirs created -----Message d'origine----- De?: boxbackup-dev-bounces at boxbackup.org [mailto:boxbackup-dev-bounces at boxbackup.org] De la part de Chris Wilson Envoy??: dimanche 11 novembre 2012 20:24 ??: Box Backup Developers List Objet?: Re: [Boxbackup-dev] [common/Windows] logging to console Hi Arnaud, On Sun, 11 Nov 2012, bb at grandville.net wrote: > Some of the error message reported to the console are system's > localized messages (with accents in my case) and not UTF8 encoded as > expected. I wrote a function to report correctly this kind of text > stream to the console My patch is attached Thanks for the patch, but I suspect this is not the correct fix. The aim of these conversions is to fix filename reporting, which should always be in UTF-8, because we convert all other encodings to UTF-8 in emu.cpp and only use UTF-8 internally. I'd like to see examples of the kinds of messages you report, and also what effect this has on log messages with filenames containing accents. I suspect that the latter would be broken by the patch. Perhaps we need to convert the sources of those messages into UTF-8 as well, e.g. GetLastError(), but I'd need to see the messages to understand whether that's the case. Cheers, Chris. -- _____ __ _ \ __/ / ,__(_)_ | Chris Wilson Cambs UK | / (_/ ,\/ _/ /_ \ | Security/C/C++/Java/Ruby/Perl/SQL Developer | \__/_/_/_//_/___/ | We are GNU : free your mind & your software | _______________________________________________ Boxbackup-dev mailing list Boxbackup-dev at boxbackup.org http://lists.boxbackup.org/cgi-bin/mailman/listinfo/boxbackup-dev From chris at qwirx.com Mon Nov 12 10:12:27 2012 From: chris at qwirx.com (Chris Wilson) Date: Mon, 12 Nov 2012 10:12:27 +0000 (GMT) Subject: [Boxbackup-dev] [bbstoreaccounts] folder remove In-Reply-To: <009d01cdbfea$e95b4e20$bc11ea60$@grandville.net> References: <009d01cdbfea$e95b4e20$bc11ea60$@grandville.net> Message-ID: Hi Arnaud, On Sun, 11 Nov 2012, bb at grandville.net wrote: > I?m using the stunning boxbackup on Windows platform, and I found a > problem when deleting an account, > > C:\dev\boxbackup\Release>bbstoreaccounts -c C:\dev\boxbackup\conf\boxbackup\bbstored.conf delete? 101 yes Please note that we explicitly DO NOT support running the Box Backup server on Windows. The only reason that it exists is to help run tests for the client. It is in no way validated, does not support multiple users and might eat your data or your kittens. > In fact, the user?s folder contains an open file write.lock so before > deleting his folders wouldn?t be mandatory to close this file ?!?! I > take this opportunity to add a notice at the end of the DeleteAccount to > validate the successful completion of the call. My proposed patch > (bbstoreaccounts.cpp.patch) is attached. Thanks for the report, I'll have a look at it. > The command to recursively delete windows folder would be ? > > rd /s /q C:\dev\boxbackup\Data\Single\backup\00000101 > > I wrote a rmdir function specific to windows in the second patch and I > reorder emu_stat struct from large to narrow items) to reduce the data > alignment. Thanks for the patch as well, I'll have a look at it. Cheers, Chris. -- _____ __ _ \ __/ / ,__(_)_ | Chris Wilson Cambs UK | / (_/ ,\/ _/ /_ \ | Security/C/C++/Java/Ruby/Perl/SQL Developer | \__/_/_/_//_/___/ | We are GNU : free your mind & your software | From chris at qwirx.com Mon Nov 12 10:15:30 2012 From: chris at qwirx.com (Chris Wilson) Date: Mon, 12 Nov 2012 10:15:30 +0000 (GMT) Subject: [Boxbackup-dev] [common/Windows] logging to console In-Reply-To: <001201cdc055$eb373770$c1a5a650$@grandville.net> References: <00b501cdc013$554eed80$ffecc880$@grandville.net> <001201cdc055$eb373770$c1a5a650$@grandville.net> Message-ID: Hi Arnaud, On Sun, 11 Nov 2012, bb at grandville.net wrote: > I understand what you mean, but I found another problem. To validate your > suspicion, I tried to use a root backup location folder name with some > accents :-)). The call to the windows version of BackupDaemon::statfs at 2368 > failed because Location.mPath must be UTF8 encoded but the > rConfig.GetKeyValue("Path") failed when it try to read my bbackupd.conf file > recorded in UTF8 format in place of the standard ASCII !?!? > > INFO: Opening connection to server 'localhost'... > INFO: Connection made, login successful > WARNING: Exception thrown: CommonException(OSFileError) (Failed to stat path > 'C:\Voix ambigu?' for location 'temp': No such file or directory (2)) at > c:\dev\boxbackup\bin\bbackupd\backupdaemon.cpp:2375 I'd need to see the exact way that "Path = C:\Voix ambigu?" is encoded in your configuration file, and the characters in the filename on disk. Cheers, Chris. -- _____ __ _ \ __/ / ,__(_)_ | Chris Wilson Cambs UK | / (_/ ,\/ _/ /_ \ | Security/C/C++/Java/Ruby/Perl/SQL Developer | \__/_/_/_//_/___/ | We are GNU : free your mind & your software | From contact at grandville.net Sun Nov 11 21:56:27 2012 From: contact at grandville.net (Arnaud Grandville) Date: Sun, 11 Nov 2012 22:56:27 +0100 Subject: [Boxbackup-dev] [WinNamedPipeListener] Everyone access In-Reply-To: <00d701cdc03d$f4cdabb0$de690310$@grandville.net> References: <00d701cdc03d$f4cdabb0$de690310$@grandville.net> Message-ID: <001601cdc057$67209a10$3561ce30$@grandville.net> Hi, I propose some modifications to the pipe's access right to allow standard users to connect to the pipe even if it's created by the user SYSTEM The second modification concern the remote pipe connection (\\\pipe\). Regards Arnaud -------------- next part -------------- A non-text attachment was scrubbed... Name: WinNamedPipe.patch Type: application/octet-stream Size: 2950 bytes Desc: not available URL: From chris at qwirx.com Mon Nov 12 22:14:53 2012 From: chris at qwirx.com (Chris Wilson) Date: Mon, 12 Nov 2012 22:14:53 +0000 (GMT) Subject: [Boxbackup-dev] [WinNamedPipeListener] Everyone access In-Reply-To: <001601cdc057$67209a10$3561ce30$@grandville.net> References: <00d701cdc03d$f4cdabb0$de690310$@grandville.net> <001601cdc057$67209a10$3561ce30$@grandville.net> Message-ID: Hi Arnaud, On Sun, 11 Nov 2012, Arnaud Grandville wrote: > I propose some modifications to the pipe's access right to allow > standard users to connect to the pipe even if it's created by the user > SYSTEM The second modification concern the remote pipe connection > (\\\pipe\). Are you sure that's safe? Both remote access and connections by other users are potentially dangerous. All connections should be limited to Administrators at least. Do you patches do that? Cheers, Chris. -- _____ __ _ \ __/ / ,__(_)_ | Chris Wilson Cambs UK | / (_/ ,\/ _/ /_ \ | Security/C/C++/Java/Ruby/Perl/SQL Developer | \__/_/_/_//_/___/ | We are GNU : free your mind & your software | From lists at invis.net Mon Nov 12 22:42:35 2012 From: lists at invis.net (Charles Lecklider) Date: Mon, 12 Nov 2012 22:42:35 +0000 Subject: [Boxbackup-dev] [WinNamedPipeListener] Everyone access In-Reply-To: References: <00d701cdc03d$f4cdabb0$de690310$@grandville.net> <001601cdc057$67209a10$3561ce30$@grandville.net> Message-ID: <50A17B5B.50005@invis.net> On 12/11/2012 22:14, Chris Wilson wrote: > Are you sure that's safe? Both remote access and connections by other > users are potentially dangerous. All connections should be limited to > Administrators at least. Do you patches do that? Agreed. In my (fabled) branch I created a BB admin group and set the perms for that, and then deliberately didn't disable remote access as I can see that being quite handy for remote admin. I don't think this patch is a good idea. -C From chris at qwirx.com Mon Nov 12 22:54:51 2012 From: chris at qwirx.com (Chris Wilson) Date: Mon, 12 Nov 2012 22:54:51 +0000 (GMT) Subject: [Boxbackup-dev] [WinNamedPipeListener] Everyone access In-Reply-To: <50A17B5B.50005@invis.net> References: <00d701cdc03d$f4cdabb0$de690310$@grandville.net> <001601cdc057$67209a10$3561ce30$@grandville.net> <50A17B5B.50005@invis.net> Message-ID: Hey Charles, On Mon, 12 Nov 2012, Charles Lecklider wrote: > On 12/11/2012 22:14, Chris Wilson wrote: >> Are you sure that's safe? Both remote access and connections by other >> users are potentially dangerous. All connections should be limited to >> Administrators at least. Do you patches do that? > > Agreed. In my (fabled) branch I created a BB admin group and set the > perms for that, and then deliberately didn't disable remote access as I > can see that being quite handy for remote admin. I don't think this > patch is a good idea. Good to see that you're still with us :) How about we start to merge that fabled branch of yore? :) Cheers, Chris. -- _____ __ _ \ __/ / ,__(_)_ | Chris Wilson Cambs UK | / (_/ ,\/ _/ /_ \ | Security/C/C++/Java/Ruby/Perl/SQL Developer | \__/_/_/_//_/___/ | We are GNU : free your mind & your software | From lists at invis.net Mon Nov 12 23:50:34 2012 From: lists at invis.net (Charles Lecklider) Date: Mon, 12 Nov 2012 23:50:34 +0000 Subject: [Boxbackup-dev] [bbstoreaccounts] folder remove In-Reply-To: <009d01cdbfea$e95b4e20$bc11ea60$@grandville.net> References: <009d01cdbfea$e95b4e20$bc11ea60$@grandville.net> Message-ID: <50A18B4A.8010709@invis.net> On 11/11/2012 08:59, bb at grandville.net wrote: > I wrote a rmdir function specific to windows in the second patch Using the Shell API in something designed to be run as a service is usually not a good idea; assuming for a moment that it's warranted in this case (which I'm not at all convinced about), the name of the function should reflect its recursive nature lest some unfortunate future developer get a nasty surprise.... > and I reorder emu_stat struct from large to narrow items) to reduce > the data alignment. struct stat is pretty much standard across all platforms. Re-ordering it to save a few bytes is, IMO, not a good enough reason to break compatibility. -C From chris at qwirx.com Mon Nov 12 23:52:54 2012 From: chris at qwirx.com (Chris Wilson) Date: Mon, 12 Nov 2012 23:52:54 +0000 (GMT) Subject: [Boxbackup-dev] [bbstoreaccounts] folder remove In-Reply-To: <50A18B4A.8010709@invis.net> References: <009d01cdbfea$e95b4e20$bc11ea60$@grandville.net> <50A18B4A.8010709@invis.net> Message-ID: Hi Charles, On Mon, 12 Nov 2012, Charles Lecklider wrote: >> and I reorder emu_stat struct from large to narrow items) to reduce >> the data alignment. > > struct stat is pretty much standard across all platforms. Re-ordering it > to save a few bytes is, IMO, not a good enough reason to break > compatibility. In this case we're using a non-standard struct stat anyway, because we need a larger field for st_ino than the one provided by Windows, so moving things around doesn't actually break anything that wasn't already broken. But it does feel like a micro-optimisation to me. Cheers, Chris. -- _____ __ _ \ __/ / ,__(_)_ | Chris Wilson Cambs UK | / (_/ ,\/ _/ /_ \ | Security/C/C++/Java/Ruby/Perl/SQL Developer | \__/_/_/_//_/___/ | We are GNU : free your mind & your software | From lists at invis.net Tue Nov 13 00:02:36 2012 From: lists at invis.net (Charles Lecklider) Date: Tue, 13 Nov 2012 00:02:36 +0000 Subject: [Boxbackup-dev] [bbstoreaccounts] folder remove In-Reply-To: References: <009d01cdbfea$e95b4e20$bc11ea60$@grandville.net> <50A18B4A.8010709@invis.net> Message-ID: <50A18E1C.3090908@invis.net> On 12/11/2012 23:52, Chris Wilson wrote: > In this case we're using a non-standard struct stat anyway, because we > need a larger field for st_ino than the one provided by Windows, so > moving things around doesn't actually break anything that wasn't already > broken. But it does feel like a micro-optimisation to me. I was only talking about the order - the sizes of fields are different on many platforms. -C From bb at grandville.net Fri Nov 16 06:20:36 2012 From: bb at grandville.net (bb at grandville.net) Date: Fri, 16 Nov 2012 07:20:36 +0100 Subject: [Boxbackup-dev] condition always true ?!?! Message-ID: <000901cdc3c2$7ec7afa0$7c570ee0$@grandville.net> opendir at boxbackup\lib\win32\emu.cpp ... std::string dirName(name); //append a '\' win32 findfirst is sensitive to this if (dirName[dirName.size()-1] != '\\' || dirName[dirName.size()-1] != '/') <=== always true { dirName += '\\'; } ... From bb at grandville.net Fri Nov 16 06:37:24 2012 From: bb at grandville.net (bb at grandville.net) Date: Fri, 16 Nov 2012 07:37:24 +0100 Subject: [Boxbackup-dev] useless free(null) Message-ID: <000b01cdc3c4$d799ce40$86cd6ac0$@grandville.net> emu_chmod at boxbackup\lib\win32\emu.cpp if (pBuffer == NULL) { // error already logged by ConvertUtf8ToWideString() free(pBuffer); return -1; } Arnaud From trac at boxbackup.org Sat Nov 17 12:00:00 2012 From: trac at boxbackup.org (trac at boxbackup.org) Date: Sat, 17 Nov 2012 12:00:00 +0000 (GMT) Subject: [Boxbackup-dev] Current open tickets Message-ID: <20121117120001.17BBF198848@www.boxbackup.org> Note: to view an indiviual ticket, use: https://www.boxbackup.org/trac/ticket/(number) The following is a listing of current problems submitted by Box Backup users. These represent problem reports covering all versions including experimental development code and obsolete releases. S Ticket Owner Component Summary - ------ ------ ------------- ------------------------------------------------------------ n 4 martin box libraries Port Box Backup to AIX n 6 box libraries Contribute code: SMTP client, HTTP server, Database drivers, n 7 box libraries Improve restore speed on local repositories n 8 chris box libraries Improve handling of directories with many files n 13 chris bbackupd Fix file locking on Windows n 14 chris bbackupd Fix large file issues on Windows n 16 chris bbackupquery Restore deleted directories may fail a 17 chris bbackupquery List files using wildcards a 20 chris bbackupctl bbackupctl reload reports prior settings n 45 ben bbackupd File diff performance patch (reduced disk IO and wall time n 46 chris bbackupd bbackupd only ever saves reverse diffs, corrupted files on s n 47 chris bbackupd Account numbers greater than 2^31 (0x7fffffff) do not work c n 48 chris bbackupd Locations that don't exist on first run are never tried agai n 49 chris bbackupd ID map (rename tracking) broken since [288] n 50 chris bbackupquery No way to capture stderr under Windows n 51 chris bbackupd No way to force bbackupd to re-upload files under Windows n 52 chris bbackupd Unable to control the maintenance of old vs. deleted files n 53 chris bbackupd Comparing root directory locations does not work under Windo n 54 chris bbackupd Locations not found on disk (e.g. unmounted filesystems) can n 55 chris bbackupd Should store and preserve directory timestamps n 56 bbackupquery Windows: User can polute restore directory by adding a trail n 57 ben bbackupd Create boxbackupresume and other files in DataDirectory n 58 ben bbackupd bbackupquery connection timeout results in error message, sh n 59 ben bbackupd Inconsistent treatment of NotifyScript between Box Backup an n 60 ben bbstored Case sensitivity between bbstored-certs and Boxi n 62 ben bbackupd Backups of deleted files may be removed very quickly by hous a 63 chris bbackupd Support hard links in directories n 65 ben bbackupd Some file attributes under Windows are not backed up n 66 ben bbackupd Windows ACL support a 67 chris bbackupd Add mutexes for Win32 (client) to enable detection of runnin n 68 ben bbackupd More details when ReadError occurs n 69 bbackupquery CRTL+C closes bbackupquery console window under Windows n 70 ben bbackupd Non-ascii character cause error for notifyscript path n 71 ben bbackupd bbackupd.log gets randomly deleted n 72 test suite Patch intercept.h and intercept.c for NetBSD 4 and 5 n 74 bbackupquery Reconnect bbackupquery automatically, or at least have a "co n 75 ben bbackupd Restore times wrong, times on store correct n 77 ben bbackupd Min verbosity switch broken for bbackupquery a 78 chris bbackupd Error level for "bbackupquery.exe -u" 39 tickets total. From bb at grandville.net Sun Nov 18 22:33:17 2012 From: bb at grandville.net (bb at grandville.net) Date: Sun, 18 Nov 2012 23:33:17 +0100 Subject: [Boxbackup-dev] [common/Windows] logging to console In-Reply-To: References: <00b501cdc013$554eed80$ffecc880$@grandville.net> <001201cdc055$eb373770$c1a5a650$@grandville.net> Message-ID: <002001cdc5dc$b5223f90$1f66beb0$@grandville.net> Hi Chris, Another test with bbackup.conf ascii encoded with this content: ... BackupLocations { temp { Path = C:\Donn?es ? sauvegarder } } ... The configuration parser will append this key pair (configuration.cpp at 346) std::string key(line.substr(0, keyend)); ->key=Path std::string value(line.substr(valuestart)); ->value= C:\Donn?es ? sauvegarder rConfig.AddKeyValue(key, value); it may be useful to add a non-conditional translation of 'value' to utf8 (ConvertToUtf8(value, strUTF8value, GetACP())) before adding it to rConfig ? if my bbackup.conf is utf8 encoded, parameters becomes Path C:\Donn??es ?? sauvegarder But you'll have to deal with the Byte order mask appended by the windows text editor at the beginning of the file 000000 ef bb bf 0d 0a 53 74 6f 72 65 48 6f 73 74 6e 61 ???StoreHostna 000010 6d 65 20 3d 20 6c 6f 63 61 6c 68 6f 73 74 0d 0a me = localhost 000020 41 63 63 6f 75 6e 74 4e 75 6d 62 65 72 20 3d 20 AccountNumber = Arnaud -----Message d'origine----- De?: boxbackup-dev-bounces at boxbackup.org [mailto:boxbackup-dev-bounces at boxbackup.org] De la part de Chris Wilson Envoy??: lundi 12 novembre 2012 11:16 ??: Box Backup Developers List Objet?: Re: [Boxbackup-dev] [common/Windows] logging to console Hi Arnaud, On Sun, 11 Nov 2012, bb at grandville.net wrote: > I understand what you mean, but I found another problem. To validate > your suspicion, I tried to use a root backup location folder name with > some accents :-)). The call to the windows version of > BackupDaemon::statfs at 2368 failed because Location.mPath must be UTF8 > encoded but the > rConfig.GetKeyValue("Path") failed when it try to read my > bbackupd.conf file recorded in UTF8 format in place of the standard ASCII !?!? > > INFO: Opening connection to server 'localhost'... > INFO: Connection made, login successful > WARNING: Exception thrown: CommonException(OSFileError) (Failed to > stat path 'C:\Voix ambigu?' for location 'temp': No such file or > directory (2)) at > c:\dev\boxbackup\bin\bbackupd\backupdaemon.cpp:2375 I'd need to see the exact way that "Path = C:\Voix ambigu?" is encoded in your configuration file, and the characters in the filename on disk. Cheers, Chris. -- _____ __ _ \ __/ / ,__(_)_ | Chris Wilson Cambs UK | / (_/ ,\/ _/ /_ \ | Security/C/C++/Java/Ruby/Perl/SQL Developer | \__/_/_/_//_/___/ | We are GNU : free your mind & your software | From chris at qwirx.com Mon Nov 19 00:24:23 2012 From: chris at qwirx.com (Chris Wilson) Date: Mon, 19 Nov 2012 00:24:23 +0000 (GMT) Subject: [Boxbackup-dev] [common/Windows] logging to console In-Reply-To: <002001cdc5dc$b5223f90$1f66beb0$@grandville.net> References: <00b501cdc013$554eed80$ffecc880$@grandville.net> <001201cdc055$eb373770$c1a5a650$@grandville.net> <002001cdc5dc$b5223f90$1f66beb0$@grandville.net> Message-ID: Hi Arnaud, On Sun, 18 Nov 2012, bb at grandville.net wrote: > The configuration parser will append this key pair (configuration.cpp at 346) > > std::string key(line.substr(0, keyend)); ->key=Path > std::string value(line.substr(valuestart)); ->value= C:\Donn?es ? > sauvegarder > rConfig.AddKeyValue(key, value); > > it may be useful to add a non-conditional translation of 'value' to utf8 > (ConvertToUtf8(value, strUTF8value, GetACP())) before adding it to rConfig > ? That's annoying because if you change your system code page, or copy files from one machine to another, suddenly the paths can be unreadable. I was intending that all configuration files should be in UTF8 to avoid that, but the lack of editor support for UTF8 is also annoying. > if my bbackup.conf is utf8 encoded, parameters becomes > Path > C:\Donn??es ?? sauvegarder > > But you'll have to deal with the Byte order mask appended by the windows > text editor at the beginning of the file And that's even worse. A BOM is completely useless in a file where byte order is irrelevant. I think it must be a bug in notepad.exe. But since it's unlikely that Microsoft will change it, I think we might have to live with config files being in the current platform encoding (on Windows only) or in UTF-16 (which would require us to link with libiconv to decode them, or just skip the first three bytes if they're a Notepad BOM). Other developers and Windows users, any strong feelings or objections? Cheers, Chris. -- _____ __ _ \ __/ / ,__(_)_ | Chris Wilson Cambs UK | / (_/ ,\/ _/ /_ \ | Security/C/C++/Java/Ruby/Perl/SQL Developer | \__/_/_/_//_/___/ | We are GNU : free your mind & your software | From bb at grandville.net Mon Nov 19 07:01:54 2012 From: bb at grandville.net (bb at grandville.net) Date: Mon, 19 Nov 2012 08:01:54 +0100 Subject: [Boxbackup-dev] [common/Windows] logging to console In-Reply-To: References: <00b501cdc013$554eed80$ffecc880$@grandville.net> <001201cdc055$eb373770$c1a5a650$@grandville.net> <002001cdc5dc$b5223f90$1f66beb0$@grandville.net> Message-ID: <004601cdc623$c2f280c0$48d78240$@grandville.net> Let me clarify some things: 1. it's a very bad idea to put accents in the path ! 2. with all my windows editor (notepad, Notepad++, Write, ...), whatever the format is (ascii, utf8, utf16), the file is correct on screen, but the data stored on disk is different: - with ascii encoding, contents looks like unix file, but if it contains localized char, the 'value' must be converted from local code page to utf8 to remain usable. - with utf8 encoding, 'value' could remain as is, but file starts with tree bytes 'EF BB BF' - with utf16 (Unicode), the configuration parser must be rewrite and libiconv must be add to the project. Arnaud -----Message d'origine----- De?: boxbackup-dev-bounces at boxbackup.org [mailto:boxbackup-dev-bounces at boxbackup.org] De la part de Chris Wilson Envoy??: lundi 19 novembre 2012 01:24 ??: Box Backup Developers List Objet?: Re: [Boxbackup-dev] [common/Windows] logging to console Hi Arnaud, On Sun, 18 Nov 2012, bb at grandville.net wrote: > The configuration parser will append this key pair > (configuration.cpp at 346) > > std::string key(line.substr(0, keyend)); ->key=Path > std::string value(line.substr(valuestart)); ->value= C:\Donn?es ? > sauvegarder > rConfig.AddKeyValue(key, value); > > it may be useful to add a non-conditional translation of 'value' to > utf8 (ConvertToUtf8(value, strUTF8value, GetACP())) before adding it > to rConfig ? That's annoying because if you change your system code page, or copy files from one machine to another, suddenly the paths can be unreadable. I was intending that all configuration files should be in UTF8 to avoid that, but the lack of editor support for UTF8 is also annoying. > if my bbackup.conf is utf8 encoded, parameters becomes Path > C:\Donn??es ?? sauvegarder > > But you'll have to deal with the Byte order mask appended by the > windows text editor at the beginning of the file And that's even worse. A BOM is completely useless in a file where byte order is irrelevant. I think it must be a bug in notepad.exe. But since it's unlikely that Microsoft will change it, I think we might have to live with config files being in the current platform encoding (on Windows only) or in UTF-16 (which would require us to link with libiconv to decode them, or just skip the first three bytes if they're a Notepad BOM). Other developers and Windows users, any strong feelings or objections? Cheers, Chris. -- _____ __ _ \ __/ / ,__(_)_ | Chris Wilson Cambs UK | / (_/ ,\/ _/ /_ \ | Security/C/C++/Java/Ruby/Perl/SQL Developer | \__/_/_/_//_/___/ | We are GNU : free your mind & your software | From chris at qwirx.com Wed Nov 21 21:36:03 2012 From: chris at qwirx.com (Chris Wilson) Date: Wed, 21 Nov 2012 21:36:03 +0000 (GMT) Subject: [Boxbackup-dev] [common/Windows] logging to console In-Reply-To: <004601cdc623$c2f280c0$48d78240$@grandville.net> References: <00b501cdc013$554eed80$ffecc880$@grandville.net> <001201cdc055$eb373770$c1a5a650$@grandville.net> <002001cdc5dc$b5223f90$1f66beb0$@grandville.net> <004601cdc623$c2f280c0$48d78240$@grandville.net> Message-ID: Hi Arnaud, On Mon, 19 Nov 2012, bb at grandville.net wrote: > Let me clarify some things: > 1. it's a very bad idea to put accents in the path ! I'm not sure if you're joking :) I'm sure several of our users have accents in their paths, and we ought to support it. > 2. with all my windows editor (notepad, Notepad++, Write, ...), whatever the > format is (ascii, utf8, utf16), the file is correct on screen, but the data > stored on disk is different: > - with ascii encoding, contents looks like unix file, but if it contains > localized char, the 'value' must be converted from local code page to utf8 > to remain usable. > - with utf8 encoding, 'value' could remain as is, but file starts with tree > bytes 'EF BB BF' > - with utf16 (Unicode), the configuration parser must be rewrite and > libiconv must be add to the project. On second thoughts, I'm pretty sure that Windows includes UTF-16 conversion somewhere in the standard library, so we might not need to link with libiconv. My guess is that Windows' users expectations are that they can save files using Notepad without choosing UTF-8 encoding and things will "kind of" work (i.e. they'd better not change the system code page). So I propose as a first step to convert each value in the config file from the current system encoding to UTF-8. If that proves not to be enough, then I'll add support for detecting UTF-8 and UTF-16 files and converting them automatically. Any objections? Cheers, Chris. -- _____ __ _ \ __/ / ,__(_)_ | Chris Wilson Cambs UK | / (_/ ,\/ _/ /_ \ | Security/C/C++/Java/Ruby/Perl/SQL Developer | \__/_/_/_//_/___/ | We are GNU : free your mind & your software | From trac at boxbackup.org Sat Nov 24 12:00:01 2012 From: trac at boxbackup.org (trac at boxbackup.org) Date: Sat, 24 Nov 2012 12:00:01 +0000 (GMT) Subject: [Boxbackup-dev] Current open tickets Message-ID: <20121124120001.22FE6198193@www.boxbackup.org> Note: to view an indiviual ticket, use: https://www.boxbackup.org/trac/ticket/(number) The following is a listing of current problems submitted by Box Backup users. These represent problem reports covering all versions including experimental development code and obsolete releases. S Ticket Owner Component Summary - ------ ------ ------------- ------------------------------------------------------------ n 4 martin box libraries Port Box Backup to AIX n 6 box libraries Contribute code: SMTP client, HTTP server, Database drivers, n 7 box libraries Improve restore speed on local repositories n 8 chris box libraries Improve handling of directories with many files n 13 chris bbackupd Fix file locking on Windows n 14 chris bbackupd Fix large file issues on Windows n 16 chris bbackupquery Restore deleted directories may fail a 17 chris bbackupquery List files using wildcards a 20 chris bbackupctl bbackupctl reload reports prior settings n 45 ben bbackupd File diff performance patch (reduced disk IO and wall time n 46 chris bbackupd bbackupd only ever saves reverse diffs, corrupted files on s n 47 chris bbackupd Account numbers greater than 2^31 (0x7fffffff) do not work c n 48 chris bbackupd Locations that don't exist on first run are never tried agai n 49 chris bbackupd ID map (rename tracking) broken since [288] n 50 chris bbackupquery No way to capture stderr under Windows n 51 chris bbackupd No way to force bbackupd to re-upload files under Windows n 52 chris bbackupd Unable to control the maintenance of old vs. deleted files n 53 chris bbackupd Comparing root directory locations does not work under Windo n 54 chris bbackupd Locations not found on disk (e.g. unmounted filesystems) can n 55 chris bbackupd Should store and preserve directory timestamps n 56 bbackupquery Windows: User can polute restore directory by adding a trail n 57 ben bbackupd Create boxbackupresume and other files in DataDirectory n 58 ben bbackupd bbackupquery connection timeout results in error message, sh n 59 ben bbackupd Inconsistent treatment of NotifyScript between Box Backup an n 60 ben bbstored Case sensitivity between bbstored-certs and Boxi n 62 ben bbackupd Backups of deleted files may be removed very quickly by hous a 63 chris bbackupd Support hard links in directories n 65 ben bbackupd Some file attributes under Windows are not backed up n 66 ben bbackupd Windows ACL support a 67 chris bbackupd Add mutexes for Win32 (client) to enable detection of runnin n 68 ben bbackupd More details when ReadError occurs n 69 bbackupquery CRTL+C closes bbackupquery console window under Windows n 70 ben bbackupd Non-ascii character cause error for notifyscript path n 71 ben bbackupd bbackupd.log gets randomly deleted n 72 test suite Patch intercept.h and intercept.c for NetBSD 4 and 5 n 74 bbackupquery Reconnect bbackupquery automatically, or at least have a "co n 75 ben bbackupd Restore times wrong, times on store correct n 77 ben bbackupd Min verbosity switch broken for bbackupquery a 78 chris bbackupd Error level for "bbackupquery.exe -u" 39 tickets total.