From trac at boxbackup.org Sat Apr 3 12:00:01 2010 From: trac at boxbackup.org (trac at boxbackup.org) Date: Sat, 3 Apr 2010 12:00:01 +0100 (BST) Subject: [Boxbackup-dev] Current open tickets Message-ID: <20100403110001.E964B32605E@www.boxbackup.org> Note: to view an indiviual ticket, use: https://www.boxbackup.org/trac/ticket/(number) The following is a listing of current problems submitted by Box Backup users. These represent problem reports covering all versions including experimental development code and obsolete releases. S Ticket Owner Component Summary - ------ ------ ------------- ------------------------------------------------------------ n 4 martin box libraries Port Box Backup to AIX n 6 box libraries Contribute code: SMTP client, HTTP server, Database drivers, n 7 box libraries Improve restore speed on local repositories n 8 chris box libraries Improve handling of directories with many files n 13 chris bbackupd Fix file locking on Windows n 14 chris bbackupd Fix large file issues on Windows n 16 chris bbackupquery Restore deleted directories may fail a 17 chris bbackupquery List files using wildcards a 20 chris bbackupctl bbackupctl reload reports prior settings n 45 ben bbackupd File diff performance patch (reduced disk IO and wall time n 46 chris bbackupd bbackupd only ever saves reverse diffs, corrupted files on s n 47 chris bbackupd Account numbers greater than 2^31 (0x7fffffff) do not work c n 48 chris bbackupd Locations that don't exist on first run are never tried agai n 49 chris bbackupd ID map (rename tracking) broken since [288] n 50 chris bbackupquery No way to capture stderr under Windows n 51 chris bbackupd No way to force bbackupd to re-upload files under Windows n 52 chris bbackupd Unable to control the maintenance of old vs. deleted files n 53 chris bbackupd Comparing root directory locations does not work under Windo n 54 chris bbackupd Locations not found on disk (e.g. unmounted filesystems) can n 55 chris bbackupd Should store and preserve directory timestamps n 56 bbackupquery Windows: User can polute restore directory by adding a trail n 57 ben bbackupd Create boxbackupresume and other files in DataDirectory n 58 ben bbackupd bbackupquery connection timeout results in error message, sh n 59 ben bbackupd Inconsistent treatment of NotifyScript between Box Backup an n 60 ben bbstored Case sensitivity between bbstored-certs and Boxi n 61 ben bbackupd File statistics: Log the number of uploaded files in additio n 62 ben bbackupd Backups of deleted files may be removed very quickly by hous a 63 chris bbackupd Support hard links in directories n 65 ben bbackupd Some file attributes under Windows are not backed up n 66 ben bbackupd Windows ACL support a 67 chris bbackupd Add mutexes for Win32 (client) to enable detection of runnin n 68 ben bbackupd More details when ReadError occurs n 69 bbackupquery CRTL+C closes bbackupquery console window under Windows n 70 ben bbackupd Non-ascii character cause error for notifyscript path 34 tickets total. From trac at boxbackup.org Sat Apr 10 12:00:01 2010 From: trac at boxbackup.org (trac at boxbackup.org) Date: Sat, 10 Apr 2010 12:00:01 +0100 (BST) Subject: [Boxbackup-dev] Current open tickets Message-ID: <20100410110001.613B3326049@www.boxbackup.org> Note: to view an indiviual ticket, use: https://www.boxbackup.org/trac/ticket/(number) The following is a listing of current problems submitted by Box Backup users. These represent problem reports covering all versions including experimental development code and obsolete releases. S Ticket Owner Component Summary - ------ ------ ------------- ------------------------------------------------------------ n 4 martin box libraries Port Box Backup to AIX n 6 box libraries Contribute code: SMTP client, HTTP server, Database drivers, n 7 box libraries Improve restore speed on local repositories n 8 chris box libraries Improve handling of directories with many files n 13 chris bbackupd Fix file locking on Windows n 14 chris bbackupd Fix large file issues on Windows n 16 chris bbackupquery Restore deleted directories may fail a 17 chris bbackupquery List files using wildcards a 20 chris bbackupctl bbackupctl reload reports prior settings n 45 ben bbackupd File diff performance patch (reduced disk IO and wall time n 46 chris bbackupd bbackupd only ever saves reverse diffs, corrupted files on s n 47 chris bbackupd Account numbers greater than 2^31 (0x7fffffff) do not work c n 48 chris bbackupd Locations that don't exist on first run are never tried agai n 49 chris bbackupd ID map (rename tracking) broken since [288] n 50 chris bbackupquery No way to capture stderr under Windows n 51 chris bbackupd No way to force bbackupd to re-upload files under Windows n 52 chris bbackupd Unable to control the maintenance of old vs. deleted files n 53 chris bbackupd Comparing root directory locations does not work under Windo n 54 chris bbackupd Locations not found on disk (e.g. unmounted filesystems) can n 55 chris bbackupd Should store and preserve directory timestamps n 56 bbackupquery Windows: User can polute restore directory by adding a trail n 57 ben bbackupd Create boxbackupresume and other files in DataDirectory n 58 ben bbackupd bbackupquery connection timeout results in error message, sh n 59 ben bbackupd Inconsistent treatment of NotifyScript between Box Backup an n 60 ben bbstored Case sensitivity between bbstored-certs and Boxi n 61 ben bbackupd File statistics: Log the number of uploaded files in additio n 62 ben bbackupd Backups of deleted files may be removed very quickly by hous a 63 chris bbackupd Support hard links in directories n 65 ben bbackupd Some file attributes under Windows are not backed up n 66 ben bbackupd Windows ACL support a 67 chris bbackupd Add mutexes for Win32 (client) to enable detection of runnin n 68 ben bbackupd More details when ReadError occurs n 69 bbackupquery CRTL+C closes bbackupquery console window under Windows n 70 ben bbackupd Non-ascii character cause error for notifyscript path n 71 ben bbackupd bbackupd.log gets randomly deleted 35 tickets total. From james at netinertia.co.uk Sun Apr 11 17:52:11 2010 From: james at netinertia.co.uk (James O'Gorman) Date: Sun, 11 Apr 2010 17:52:11 +0100 Subject: [Boxbackup-dev] [Box Backup-commit] #72: Patch intercept.h and intercept.c for NetBSD 4 and 5 In-Reply-To: <046.be141a95636bc20292d3217ce6e90aae@boxbackup.org> References: <046.be141a95636bc20292d3217ce6e90aae@boxbackup.org> Message-ID: <4961B433-C507-4A51-9DBC-F04EADC8B244@netinertia.co.uk> Hi, I've tested the patch and I can't see that it actually changes anything? On my test system (NetBSD/amd64 5.0.2) the results are the same with and without the patch: test-backupstore.log:FAILED: 2 tests failed (first at Test.cpp:291) test-backupstorefix.log:FAILED: 2 tests failed (first at Test.cpp:291) test-backupstorepatch.log:FAILED: 2 tests failed (first at Test.cpp:291) test-basicserver.log:FAILED: Exception caught: Connection SocketConnectError test-bbackupd.log:FAILED: 3 tests failed (first at Test.cpp:291) test-common.log:ASSERT FAILED: [spTimers] at Timer.cpp(116) test-common.log:ASSERT FAILED: [spTimers] at Timer.cpp(133) test-common.log:ASSERT FAILED: [spTimers] at Timer.cpp(73) test-common.log:ASSERT FAILED: [!spTimers] at Timer.cpp(43) test-common.log:ASSERT FAILED: [spTimers] at Timer.cpp(73) test-httpserver.log:FAILED: Exception caught: Common ConfigNoKey I can't see any mention anywhere of code needing to use __opendir30 instead of opendir (the man page doesn't mention it at all). Could you send the results from './runtest.pl ALL debug'? I did manage to fix one thing on NetBSD though; nmake doesn't support the -C flag so I've committed a fix to cd into the Makefile directory rather than using -C. James On 11 Apr 2010, at 02:53, Box Backup wrote: > #72: Patch intercept.h and intercept.c for NetBSD 4 and 5 > -----------------------+---------------------------------------------------- > Reporter: joseyluis | Type: defect > Status: new | Priority: normal > Milestone: | Component: test suite > Version: trunk | Keywords: > -----------------------+---------------------------------------------------- > With this test the test suite works with NetBSD 4 and 5 probably, I have > tested it with NetBSD 4. Previously raidfile and bbackupd test didn't > work. > > The problem is that in NetBSD 4/5 the readdir and openddir functions > in libc library are called: __readdir30 and __opendir30. > readdir and closedir exist for compatibility with these functions in > NetBSD 3 and previous versions. > > There was a change in the ABI between the 3 and 4 version for use > 64 bits inodes in kernel instead of 32 bits. > > > > > > --- intercept.h.orig 2010-04-04 01:58:54.000000000 +0200 > +++ intercept.h 2010-04-04 02:00:33.000000000 +0200 > @@ -13,6 +13,18 @@ > > #include > > +#ifdef __NetBSD__ //__NetBSD_Version__ is defined in sys/param.h > +#include > +#endif > + > +#if defined __NetBSD_Version__ && __NetBSD_Version__ >= 399000800 > //3.99.8 vers. > +#define FUNC_OPENDIR "__opendir30" > +#define FUNC_READDIR "__readdir30" > +#else > +#define FUNC_OPENDIR "opendir" > +#define FUNC_READDIR "readdir" > +#endif > + > #include > #include > > --- intercept.cpp.orig 2010-04-04 01:58:54.000000000 +0200 > +++ intercept.cpp 2010-04-04 02:00:34.000000000 +0200 > @@ -514,7 +514,7 @@ > { > if (opendir_real == NULL) > { > - opendir_real = (opendir_t*)find_function("opendir"); > + opendir_real = (opendir_t*)find_function(FUNC_OPENDIR); > } > > if (opendir_real == NULL) > @@ -547,7 +547,7 @@ > > if (readdir_real == NULL) > { > - readdir_real = (readdir_t*)find_function("readdir"); > + readdir_real = (readdir_t*)find_function(FUNC_READDIR); > } > > if (readdir_real == NULL) > > -- > Ticket URL: > Box Backup > An open source, completely automatic on-line backup system for UNIX. > _______________________________________________ > Boxbackup-commit mailing list > Boxbackup-commit at boxbackup.org > http://lists.boxbackup.org/cgi-bin/mailman/listinfo/boxbackup-commit > From joseyluis at gmail.com Tue Apr 13 02:07:35 2010 From: joseyluis at gmail.com (Jose Luis Rodriguez Garcia) Date: Tue, 13 Apr 2010 03:07:35 +0200 Subject: [Boxbackup-dev] [Box Backup-commit] #72: Patch intercept.h and intercept.c for NetBSD 4 and 5 In-Reply-To: <4961B433-C507-4A51-9DBC-F04EADC8B244@netinertia.co.uk> References: <046.be141a95636bc20292d3217ce6e90aae@boxbackup.org> <4961B433-C507-4A51-9DBC-F04EADC8B244@netinertia.co.uk> Message-ID: On Sun, Apr 11, 2010 at 6:52 PM, James O'Gorman wrote: > Hi, > > I've tested the patch and I can't see that it actually changes anything? On my test system (NetBSD/amd64 5.0.2) the results are the same with and without the patch: > > test-backupstore.log:FAILED: 2 tests failed (first at Test.cpp:291) > test-backupstorefix.log:FAILED: 2 tests failed (first at Test.cpp:291) > test-backupstorepatch.log:FAILED: 2 tests failed (first at Test.cpp:291) > test-basicserver.log:FAILED: Exception caught: Connection SocketConnectError > test-bbackupd.log:FAILED: 3 tests failed (first at Test.cpp:291) > test-common.log:ASSERT FAILED: [spTimers] at Timer.cpp(116) > test-common.log:ASSERT FAILED: [spTimers] at Timer.cpp(133) > test-common.log:ASSERT FAILED: [spTimers] at Timer.cpp(73) > test-common.log:ASSERT FAILED: [!spTimers] at Timer.cpp(43) > test-common.log:ASSERT FAILED: [spTimers] at Timer.cpp(73) > test-httpserver.log:FAILED: Exception caught: Common ConfigNoKey > > I can't see any mention anywhere of code needing to use __opendir30 instead of opendir (the man page doesn't mention it at all). Could you send the results from './runtest.pl ALL debug'? Search for RENAME in /usr/include/dirent.h: $ grep RENAME /usr/include/dirent.h DIR *opendir(const char *) __RENAME(__opendir30); struct dirent *readdir(DIR *) __RENAME(__readdir30); struct dirent ** __restrict) __RENAME(__readdir_r30); DIR *__opendir2(const char *, int) __RENAME(__opendir230); __RENAME(__scandir30); int getdents(int, char *, size_t) __RENAME(__getdents30); $ It isn't documented and it is ok. The functions __opendir30, etc are not intented for be used directly. If you follow the macro: DIR *opendir(const char *) __RENAME(__opendir30); It will be substitued as this text: DIR *opendir(const char*) asm("__opendir30"); It is a extension of gcc, and it changes all the references from opendir to __opendir30. All the code that you write and compile using opendir will be substitued as _opendir30 http://gcc.gnu.org/onlinedocs/gcc-4.3.4/gcc/Asm-Labels.html#Asm-Labels Look also 3.2.6 in: http://www.netbsd.org/docs/internals/en/chap-processes.html#traps_syscalls A old binary program will do reference to opendir, not to __opendir30, and it will work with the old abi. The only problem with this solution is when you search the symbol in the library by its name as intercept.cpp: result = dlsym(RTLD_NEXT, pName); gcc can't identify the reference to opendir and it searches for the old implementation of readdir. I attach the outputs of runtest.pl ALL debug. With the patch all the tests sucess Without the patch: common: PASSED crypto: PASSED compress: PASSED raidfile: FAILED: Exception caught: RaidFile OSError basicserver: PASSED backupstore: PASSED backupstorefix: PASSED backupstorepatch: PASSED backupdiff: PASSED bbackupd: FAILED: 1 tests failed (first at testbbackupd.cpp:937) httpserver: PASSED Have you defined PLATFORM_CLIB_FNS_INTERCEPTION_IMPOSSIBLE ? Try to clean the sources after of apply the patches. It seems that NetBSD 5 has more errors that NetBSD 4. My NetBSD Version is the last NetBSD 4 stable from cvs(2-3 weeks ago). > > I did manage to fix one thing on NetBSD though; nmake doesn't support the -C flag so I've committed a fix to cd into the Makefile directory rather than using -C. Yes I have observed that make failed and I had to do a cd. Thanks for the patch. I think that it also fails when you do a make clean. > > James > > On 11 Apr 2010, at 02:53, Box Backup wrote: > >> #72: Patch intercept.h and intercept.c for NetBSD 4 and 5 >> -----------------------+---------------------------------------------------- >> Reporter: joseyluis | Type: defect >> Status: new | Priority: normal >> Milestone: | Component: test suite >> Version: trunk | Keywords: >> -----------------------+---------------------------------------------------- >> With this test the test suite works with NetBSD 4 and 5 probably, I have >> tested it with NetBSD 4. Previously raidfile and bbackupd test didn't >> work. >> >> The problem is that in NetBSD 4/5 the readdir and openddir functions >> in libc library are called: __readdir30 and __opendir30. >> readdir and closedir exist for compatibility with these functions in >> NetBSD 3 and previous versions. >> >> There was a change in the ABI between the 3 and 4 version for use >> 64 bits inodes in kernel instead of 32 bits. >> >> >> >> >> >> --- intercept.h.orig 2010-04-04 01:58:54.000000000 +0200 >> +++ intercept.h 2010-04-04 02:00:33.000000000 +0200 >> @@ -13,6 +13,18 @@ >> >> #include >> >> +#ifdef __NetBSD__ //__NetBSD_Version__ is defined in sys/param.h >> +#include >> +#endif >> + >> +#if defined __NetBSD_Version__ && __NetBSD_Version__ >= 399000800 >> //3.99.8 vers. >> +#define FUNC_OPENDIR "__opendir30" >> +#define FUNC_READDIR "__readdir30" >> +#else >> +#define FUNC_OPENDIR "opendir" >> +#define FUNC_READDIR "readdir" >> +#endif >> + >> #include >> #include >> >> --- intercept.cpp.orig 2010-04-04 01:58:54.000000000 +0200 >> +++ intercept.cpp 2010-04-04 02:00:34.000000000 +0200 >> @@ -514,7 +514,7 @@ >> { >> if (opendir_real == NULL) >> { >> - opendir_real = (opendir_t*)find_function("opendir"); >> + opendir_real = (opendir_t*)find_function(FUNC_OPENDIR); >> } >> >> if (opendir_real == NULL) >> @@ -547,7 +547,7 @@ >> >> if (readdir_real == NULL) >> { >> - readdir_real = (readdir_t*)find_function("readdir"); >> + readdir_real = (readdir_t*)find_function(FUNC_READDIR); >> } >> >> if (readdir_real == NULL) >> >> -- >> Ticket URL: >> Box Backup >> An open source, completely automatic on-line backup system for UNIX. >> _______________________________________________ >> Boxbackup-commit mailing list >> Boxbackup-commit at boxbackup.org >> http://lists.boxbackup.org/cgi-bin/mailman/listinfo/boxbackup-commit >> > > From joseyluis at gmail.com Tue Apr 13 02:17:45 2010 From: joseyluis at gmail.com (Jose Luis Rodriguez Garcia) Date: Tue, 13 Apr 2010 03:17:45 +0200 Subject: [Boxbackup-dev] [Box Backup-commit] #72: Patch intercept.h and intercept.c for NetBSD 4 and 5 In-Reply-To: References: <046.be141a95636bc20292d3217ce6e90aae@boxbackup.org> <4961B433-C507-4A51-9DBC-F04EADC8B244@netinertia.co.uk> Message-ID: On Tue, Apr 13, 2010 at 3:07 AM, Jose Luis Rodriguez Garcia wrote: > On Sun, Apr 11, 2010 at 6:52 PM, James O'Gorman wrote: >> Hi, >> >> I've tested the patch and I can't see that it actually changes anything? On my test system (NetBSD/amd64 5.0.2) the results are the same with and without the patch: >> >> test-backupstore.log:FAILED: 2 tests failed (first at Test.cpp:291) >> test-backupstorefix.log:FAILED: 2 tests failed (first at Test.cpp:291) >> test-backupstorepatch.log:FAILED: 2 tests failed (first at Test.cpp:291) >> test-basicserver.log:FAILED: Exception caught: Connection SocketConnectError >> test-bbackupd.log:FAILED: 3 tests failed (first at Test.cpp:291) >> test-common.log:ASSERT FAILED: [spTimers] at Timer.cpp(116) >> test-common.log:ASSERT FAILED: [spTimers] at Timer.cpp(133) >> test-common.log:ASSERT FAILED: [spTimers] at Timer.cpp(73) >> test-common.log:ASSERT FAILED: [!spTimers] at Timer.cpp(43) >> test-common.log:ASSERT FAILED: [spTimers] at Timer.cpp(73) >> test-httpserver.log:FAILED: Exception caught: Common ConfigNoKey >> >> I can't see any mention anywhere of code needing to use __opendir30 instead of opendir (the man page doesn't mention it at all). Could you send the results from './runtest.pl ALL debug'? > > Search for RENAME in /usr/include/dirent.h: > $ grep RENAME /usr/include/dirent.h > > DIR *opendir(const char *) __RENAME(__opendir30); > struct dirent *readdir(DIR *) __RENAME(__readdir30); > struct dirent ** __restrict) __RENAME(__readdir_r30); > DIR *__opendir2(const char *, int) __RENAME(__opendir230); > __RENAME(__scandir30); > int getdents(int, char *, size_t) __RENAME(__getdents30); > $ > > It isn't documented and it is ok. The functions __opendir30, etc are > not intented for > be used directly. > > If you follow the macro: DIR *opendir(const char *) __RENAME(__opendir30); > It will be substitued as this text: > DIR *opendir(const char*) asm("__opendir30"); > > It is a extension of gcc, and it changes all the references from > opendir to __opendir30. > All the code that you write and compile using opendir will be > substitued as _opendir30 > http://gcc.gnu.org/onlinedocs/gcc-4.3.4/gcc/Asm-Labels.html#Asm-Labels > > Look also 3.2.6 in: > http://www.netbsd.org/docs/internals/en/chap-processes.html#traps_syscalls > > A old binary program will do reference to opendir, not to __opendir30, > and it will work with the old abi. > The only problem with this solution is when you search the symbol in > the library by its > name as intercept.cpp: > > result = dlsym(RTLD_NEXT, pName); > > gcc can't identify the reference to opendir and it searches for the > old implementation of readdir. > > I attach the outputs of runtest.pl ALL debug. > With the patch all the tests sucess > > Without the patch: > common: PASSED > crypto: PASSED > compress: PASSED > raidfile: FAILED: Exception caught: RaidFile OSError > basicserver: PASSED > backupstore: PASSED > backupstorefix: PASSED > backupstorepatch: PASSED > backupdiff: PASSED > bbackupd: FAILED: 1 tests failed (first at testbbackupd.cpp:937) > httpserver: PASSED > > Have you defined PLATFORM_CLIB_FNS_INTERCEPTION_IMPOSSIBLE ? > Try to clean the sources after of apply the patches. > > It seems that NetBSD 5 has more errors that NetBSD 4. > > > > My NetBSD Version is the last NetBSD 4 stable from cvs(2-3 weeks ago). > >> >> I did manage to fix one thing on NetBSD though; nmake doesn't support the -C flag so I've committed a fix to cd into the Makefile directory rather than using -C. > Yes I have observed that make failed and I had to do a cd. Thanks for the patch. > > I think that it also fails when you do a make clean. >> >> James >> >> On 11 Apr 2010, at 02:53, Box Backup wrote: >> >>> #72: Patch intercept.h and intercept.c for NetBSD 4 and 5 >>> -----------------------+---------------------------------------------------- >>> Reporter: joseyluis | Type: defect >>> Status: new | Priority: normal >>> Milestone: | Component: test suite >>> Version: trunk | Keywords: >>> -----------------------+---------------------------------------------------- >>> With this test the test suite works with NetBSD 4 and 5 probably, I have >>> tested it with NetBSD 4. Previously raidfile and bbackupd test didn't >>> work. >>> >>> The problem is that in NetBSD 4/5 the readdir and openddir functions >>> in libc library are called: __readdir30 and __opendir30. >>> readdir and closedir exist for compatibility with these functions in >>> NetBSD 3 and previous versions. >>> >>> There was a change in the ABI between the 3 and 4 version for use >>> 64 bits inodes in kernel instead of 32 bits. >>> >>> >>> >>> >>> >>> --- intercept.h.orig 2010-04-04 01:58:54.000000000 +0200 >>> +++ intercept.h 2010-04-04 02:00:33.000000000 +0200 >>> @@ -13,6 +13,18 @@ >>> >>> #include >>> >>> +#ifdef __NetBSD__ //__NetBSD_Version__ is defined in sys/param.h >>> +#include >>> +#endif >>> + >>> +#if defined __NetBSD_Version__ && __NetBSD_Version__ >= 399000800 >>> //3.99.8 vers. >>> +#define FUNC_OPENDIR "__opendir30" >>> +#define FUNC_READDIR "__readdir30" >>> +#else >>> +#define FUNC_OPENDIR "opendir" >>> +#define FUNC_READDIR "readdir" >>> +#endif >>> + >>> #include >>> #include >>> >>> --- intercept.cpp.orig 2010-04-04 01:58:54.000000000 +0200 >>> +++ intercept.cpp 2010-04-04 02:00:34.000000000 +0200 >>> @@ -514,7 +514,7 @@ >>> { >>> if (opendir_real == NULL) >>> { >>> - opendir_real = (opendir_t*)find_function("opendir"); >>> + opendir_real = (opendir_t*)find_function(FUNC_OPENDIR); >>> } >>> >>> if (opendir_real == NULL) >>> @@ -547,7 +547,7 @@ >>> >>> if (readdir_real == NULL) >>> { >>> - readdir_real = (readdir_t*)find_function("readdir"); >>> + readdir_real = (readdir_t*)find_function(FUNC_READDIR); >>> } >>> >>> if (readdir_real == NULL) >>> >>> -- >>> Ticket URL: >>> Box Backup >>> An open source, completely automatic on-line backup system for UNIX. >>> _______________________________________________ >>> Boxbackup-commit mailing list >>> Boxbackup-commit at boxbackup.org >>> http://lists.boxbackup.org/cgi-bin/mailman/listinfo/boxbackup-commit >>> >> >> > -------------- next part -------------- A non-text attachment was scrubbed... Name: runtest_ALL_debug_without_patch Type: application/octet-stream Size: 47463 bytes Desc: not available URL: -------------- next part -------------- A non-text attachment was scrubbed... Name: runtest_ALL_debug_with_patch Type: application/octet-stream Size: 370518 bytes Desc: not available URL: From joseyluis at gmail.com Fri Apr 16 15:25:02 2010 From: joseyluis at gmail.com (Jose Luis Rodriguez Garcia) Date: Fri, 16 Apr 2010 16:25:02 +0200 Subject: [Boxbackup-dev] [Box Backup-commit] #72: Patch intercept.h and intercept.c for NetBSD 4 and 5 In-Reply-To: References: <046.be141a95636bc20292d3217ce6e90aae@boxbackup.org> <4961B433-C507-4A51-9DBC-F04EADC8B244@netinertia.co.uk> Message-ID: Hi James did you read my e-mail? On Tue, Apr 13, 2010 at 3:07 AM, Jose Luis Rodriguez Garcia wrote: > On Sun, Apr 11, 2010 at 6:52 PM, James O'Gorman wrote: >> Hi, >> >> I've tested the patch and I can't see that it actually changes anything? On my test system (NetBSD/amd64 5.0.2) the results are the same with and without the patch: >> >> test-backupstore.log:FAILED: 2 tests failed (first at Test.cpp:291) >> test-backupstorefix.log:FAILED: 2 tests failed (first at Test.cpp:291) >> test-backupstorepatch.log:FAILED: 2 tests failed (first at Test.cpp:291) >> test-basicserver.log:FAILED: Exception caught: Connection SocketConnectError >> test-bbackupd.log:FAILED: 3 tests failed (first at Test.cpp:291) >> test-common.log:ASSERT FAILED: [spTimers] at Timer.cpp(116) >> test-common.log:ASSERT FAILED: [spTimers] at Timer.cpp(133) >> test-common.log:ASSERT FAILED: [spTimers] at Timer.cpp(73) >> test-common.log:ASSERT FAILED: [!spTimers] at Timer.cpp(43) >> test-common.log:ASSERT FAILED: [spTimers] at Timer.cpp(73) >> test-httpserver.log:FAILED: Exception caught: Common ConfigNoKey >> >> I can't see any mention anywhere of code needing to use __opendir30 instead of opendir (the man page doesn't mention it at all). Could you send the results from './runtest.pl ALL debug'? > > Search for RENAME in /usr/include/dirent.h: > $ grep RENAME /usr/include/dirent.h > > DIR *opendir(const char *) __RENAME(__opendir30); > struct dirent *readdir(DIR *) __RENAME(__readdir30); > struct dirent ** __restrict) __RENAME(__readdir_r30); > DIR *__opendir2(const char *, int) __RENAME(__opendir230); > __RENAME(__scandir30); > int getdents(int, char *, size_t) __RENAME(__getdents30); > $ > > It isn't documented and it is ok. The functions __opendir30, etc are > not intented for > be used directly. > > If you follow the macro: DIR *opendir(const char *) __RENAME(__opendir30); > It will be substitued as this text: > DIR *opendir(const char*) asm("__opendir30"); > > It is a extension of gcc, and it changes all the references from > opendir to __opendir30. > All the code that you write and compile using opendir will be > substitued as _opendir30 > http://gcc.gnu.org/onlinedocs/gcc-4.3.4/gcc/Asm-Labels.html#Asm-Labels > > Look also 3.2.6 in: > http://www.netbsd.org/docs/internals/en/chap-processes.html#traps_syscalls > > A old binary program will do reference to opendir, not to __opendir30, > and it will work with the old abi. > The only problem with this solution is when you search the symbol in > the library by its > name as intercept.cpp: > > result = dlsym(RTLD_NEXT, pName); > > gcc can't identify the reference to opendir and it searches for the > old implementation of readdir. > > I attach the outputs of runtest.pl ALL debug. > With the patch all the tests sucess > > Without the patch: > common: PASSED > crypto: PASSED > compress: PASSED > raidfile: FAILED: Exception caught: RaidFile OSError > basicserver: PASSED > backupstore: PASSED > backupstorefix: PASSED > backupstorepatch: PASSED > backupdiff: PASSED > bbackupd: FAILED: 1 tests failed (first at testbbackupd.cpp:937) > httpserver: PASSED > > Have you defined PLATFORM_CLIB_FNS_INTERCEPTION_IMPOSSIBLE ? > Try to clean the sources after of apply the patches. > > It seems that NetBSD 5 has more errors that NetBSD 4. > > > > My NetBSD Version is the last NetBSD 4 stable from cvs(2-3 weeks ago). > >> >> I did manage to fix one thing on NetBSD though; nmake doesn't support the -C flag so I've committed a fix to cd into the Makefile directory rather than using -C. > Yes I have observed that make failed and I had to do a cd. Thanks for the patch. > > I think that it also fails when you do a make clean. >> >> James >> >> On 11 Apr 2010, at 02:53, Box Backup wrote: >> >>> #72: Patch intercept.h and intercept.c for NetBSD 4 and 5 >>> -----------------------+---------------------------------------------------- >>> Reporter: joseyluis | Type: defect >>> Status: new | Priority: normal >>> Milestone: | Component: test suite >>> Version: trunk | Keywords: >>> -----------------------+---------------------------------------------------- >>> With this test the test suite works with NetBSD 4 and 5 probably, I have >>> tested it with NetBSD 4. Previously raidfile and bbackupd test didn't >>> work. >>> >>> The problem is that in NetBSD 4/5 the readdir and openddir functions >>> in libc library are called: __readdir30 and __opendir30. >>> readdir and closedir exist for compatibility with these functions in >>> NetBSD 3 and previous versions. >>> >>> There was a change in the ABI between the 3 and 4 version for use >>> 64 bits inodes in kernel instead of 32 bits. >>> >>> >>> >>> >>> >>> --- intercept.h.orig 2010-04-04 01:58:54.000000000 +0200 >>> +++ intercept.h 2010-04-04 02:00:33.000000000 +0200 >>> @@ -13,6 +13,18 @@ >>> >>> #include >>> >>> +#ifdef __NetBSD__ //__NetBSD_Version__ is defined in sys/param.h >>> +#include >>> +#endif >>> + >>> +#if defined __NetBSD_Version__ && __NetBSD_Version__ >= 399000800 >>> //3.99.8 vers. >>> +#define FUNC_OPENDIR "__opendir30" >>> +#define FUNC_READDIR "__readdir30" >>> +#else >>> +#define FUNC_OPENDIR "opendir" >>> +#define FUNC_READDIR "readdir" >>> +#endif >>> + >>> #include >>> #include >>> >>> --- intercept.cpp.orig 2010-04-04 01:58:54.000000000 +0200 >>> +++ intercept.cpp 2010-04-04 02:00:34.000000000 +0200 >>> @@ -514,7 +514,7 @@ >>> { >>> if (opendir_real == NULL) >>> { >>> - opendir_real = (opendir_t*)find_function("opendir"); >>> + opendir_real = (opendir_t*)find_function(FUNC_OPENDIR); >>> } >>> >>> if (opendir_real == NULL) >>> @@ -547,7 +547,7 @@ >>> >>> if (readdir_real == NULL) >>> { >>> - readdir_real = (readdir_t*)find_function("readdir"); >>> + readdir_real = (readdir_t*)find_function(FUNC_READDIR); >>> } >>> >>> if (readdir_real == NULL) >>> >>> -- >>> Ticket URL: >>> Box Backup >>> An open source, completely automatic on-line backup system for UNIX. >>> _______________________________________________ >>> Boxbackup-commit mailing list >>> Boxbackup-commit at boxbackup.org >>> http://lists.boxbackup.org/cgi-bin/mailman/listinfo/boxbackup-commit >>> >> >> > From trac at boxbackup.org Sat Apr 17 12:00:01 2010 From: trac at boxbackup.org (trac at boxbackup.org) Date: Sat, 17 Apr 2010 12:00:01 +0100 (BST) Subject: [Boxbackup-dev] Current open tickets Message-ID: <20100417110001.693AF325097@www.boxbackup.org> Note: to view an indiviual ticket, use: https://www.boxbackup.org/trac/ticket/(number) The following is a listing of current problems submitted by Box Backup users. These represent problem reports covering all versions including experimental development code and obsolete releases. S Ticket Owner Component Summary - ------ ------ ------------- ------------------------------------------------------------ n 4 martin box libraries Port Box Backup to AIX n 6 box libraries Contribute code: SMTP client, HTTP server, Database drivers, n 7 box libraries Improve restore speed on local repositories n 8 chris box libraries Improve handling of directories with many files n 13 chris bbackupd Fix file locking on Windows n 14 chris bbackupd Fix large file issues on Windows n 16 chris bbackupquery Restore deleted directories may fail a 17 chris bbackupquery List files using wildcards a 20 chris bbackupctl bbackupctl reload reports prior settings n 45 ben bbackupd File diff performance patch (reduced disk IO and wall time n 46 chris bbackupd bbackupd only ever saves reverse diffs, corrupted files on s n 47 chris bbackupd Account numbers greater than 2^31 (0x7fffffff) do not work c n 48 chris bbackupd Locations that don't exist on first run are never tried agai n 49 chris bbackupd ID map (rename tracking) broken since [288] n 50 chris bbackupquery No way to capture stderr under Windows n 51 chris bbackupd No way to force bbackupd to re-upload files under Windows n 52 chris bbackupd Unable to control the maintenance of old vs. deleted files n 53 chris bbackupd Comparing root directory locations does not work under Windo n 54 chris bbackupd Locations not found on disk (e.g. unmounted filesystems) can n 55 chris bbackupd Should store and preserve directory timestamps n 56 bbackupquery Windows: User can polute restore directory by adding a trail n 57 ben bbackupd Create boxbackupresume and other files in DataDirectory n 58 ben bbackupd bbackupquery connection timeout results in error message, sh n 59 ben bbackupd Inconsistent treatment of NotifyScript between Box Backup an n 60 ben bbstored Case sensitivity between bbstored-certs and Boxi n 61 ben bbackupd File statistics: Log the number of uploaded files in additio n 62 ben bbackupd Backups of deleted files may be removed very quickly by hous a 63 chris bbackupd Support hard links in directories n 65 ben bbackupd Some file attributes under Windows are not backed up n 66 ben bbackupd Windows ACL support a 67 chris bbackupd Add mutexes for Win32 (client) to enable detection of runnin n 68 ben bbackupd More details when ReadError occurs n 69 bbackupquery CRTL+C closes bbackupquery console window under Windows n 70 ben bbackupd Non-ascii character cause error for notifyscript path n 71 ben bbackupd bbackupd.log gets randomly deleted n 72 test suite Patch intercept.h and intercept.c for NetBSD 4 and 5 36 tickets total. From trac at boxbackup.org Sat Apr 24 12:00:01 2010 From: trac at boxbackup.org (trac at boxbackup.org) Date: Sat, 24 Apr 2010 12:00:01 +0100 (BST) Subject: [Boxbackup-dev] Current open tickets Message-ID: <20100424110001.406A1325FF4@www.boxbackup.org> Note: to view an indiviual ticket, use: https://www.boxbackup.org/trac/ticket/(number) The following is a listing of current problems submitted by Box Backup users. These represent problem reports covering all versions including experimental development code and obsolete releases. S Ticket Owner Component Summary - ------ ------ ------------- ------------------------------------------------------------ n 4 martin box libraries Port Box Backup to AIX n 6 box libraries Contribute code: SMTP client, HTTP server, Database drivers, n 7 box libraries Improve restore speed on local repositories n 8 chris box libraries Improve handling of directories with many files n 13 chris bbackupd Fix file locking on Windows n 14 chris bbackupd Fix large file issues on Windows n 16 chris bbackupquery Restore deleted directories may fail a 17 chris bbackupquery List files using wildcards a 20 chris bbackupctl bbackupctl reload reports prior settings n 45 ben bbackupd File diff performance patch (reduced disk IO and wall time n 46 chris bbackupd bbackupd only ever saves reverse diffs, corrupted files on s n 47 chris bbackupd Account numbers greater than 2^31 (0x7fffffff) do not work c n 48 chris bbackupd Locations that don't exist on first run are never tried agai n 49 chris bbackupd ID map (rename tracking) broken since [288] n 50 chris bbackupquery No way to capture stderr under Windows n 51 chris bbackupd No way to force bbackupd to re-upload files under Windows n 52 chris bbackupd Unable to control the maintenance of old vs. deleted files n 53 chris bbackupd Comparing root directory locations does not work under Windo n 54 chris bbackupd Locations not found on disk (e.g. unmounted filesystems) can n 55 chris bbackupd Should store and preserve directory timestamps n 56 bbackupquery Windows: User can polute restore directory by adding a trail n 57 ben bbackupd Create boxbackupresume and other files in DataDirectory n 58 ben bbackupd bbackupquery connection timeout results in error message, sh n 59 ben bbackupd Inconsistent treatment of NotifyScript between Box Backup an n 60 ben bbstored Case sensitivity between bbstored-certs and Boxi n 61 ben bbackupd File statistics: Log the number of uploaded files in additio n 62 ben bbackupd Backups of deleted files may be removed very quickly by hous a 63 chris bbackupd Support hard links in directories n 65 ben bbackupd Some file attributes under Windows are not backed up n 66 ben bbackupd Windows ACL support a 67 chris bbackupd Add mutexes for Win32 (client) to enable detection of runnin n 68 ben bbackupd More details when ReadError occurs n 69 bbackupquery CRTL+C closes bbackupquery console window under Windows n 70 ben bbackupd Non-ascii character cause error for notifyscript path n 71 ben bbackupd bbackupd.log gets randomly deleted n 72 test suite Patch intercept.h and intercept.c for NetBSD 4 and 5 36 tickets total.