Changes between Version 2 and Version 3 of VerifyAndRestoreBackups

Nov 25, 2006, 3:00:25 PM (12 years ago)

Moved Windows regex information to Win32Regex page


  • VerifyAndRestoreBackups

    v2 v3  
    7777Read ''bin/bbackupquery/documentation.txt'' for more details.
    79 == How to Test !RegEx on Box Backup 0.10 Windows Client ==
    81 This is a way to test that files that are supposed to be excluded from backup actually are excluded (and also to test that this test works).
    83 Windows has limited command-line utilities, so I used cygwin (a little bulky, but gives me lots of control with a familiar environment). (Note that googling for "grep for windows" and "cron for windows" shows some products that might be interesting.)  All this has to be done as a user in the Windows Administrator group.  This was done on a Windows Server 2003 client backing up to an Ubuntu Dapper store server, both Box Backup version 0.10.
    85 To confirm that the bbackupd.conf !ExcludeFilesRegex function is working, make sure that you put one or more bogus files on your client hard drive that have filenames that you would want to be excluded from backing up, for example, in a cygwin window:
    86 {{{
    87 echo "this is a Backup Service test file, please do not delete, <your name/email/phone here>" > /cygdrive/d/Data/Public/cookies.txt
    88 echo "this is a Backup Service test file, please do not delete, <your name/email/phone here>" > /cygdrive/d/Data/Public/backup_test.avi
    89 }}}
    90 You'll have to wait for these files to have been checked (and hopefully skipped) by the hourly (default) backup file scan.
    92 Open your ''bbackupd.conf'' file and copy the regex that you want to test, and paste it into a notepad window so you can experiment easier. 
    94 Here is what I typically use in my bbackupd.conf !BackupLocations, a work in progress:
    95 {{{
    96 ExcludeFilesRegex = .+\.([aA][vV][iI]|[iI][sS][oO]|[mM][pP][eE]?[3gG]|[tT][mM][pP]|[bB][aAcC][kK]|[dD][bB][kK]|[bB][kK][~!1-9]|[mMtT][bB][kK]|[oO][lL][dD]|[sS][aA][vV]|[sS][wW][pP]|[xX][lL][kK]|[cC][sS][mM]|[dD][sS][kK]|[oO][bB][jJ]|[pP][aA][rR]|[dD][bB][xX]|[dD][lL][lL])$
    97  ExcludeFilesRegex = .*\\thumbs\.db$|.*\\history\.dat$|.*\\cookies\.txt$|.*\\~.*
    98 }}}
    99 But you have to tweak them before this form of testing because we will be using cygwin's regex, not Windows' (or bbackupquery's, which may be a twinkle in some developer's eye).
    101 So, next, again in a cygwin window, I find it convenient to navigate to the Box Backup directory:
    102 {{{
    103 cd /cygdrive/c/Program\ Files/Box\ Backup/
    104 }}}
    105 Then issue your version of this command (I've abbreviated my lengthy regex here for clarity, but using the entire thing does work):
    106 {{{
    107 ./bbackupquery "list -odtsr" quit | egrep -e '.+\.([aA][vV][iI]|[bB][aAcC][kK]|qbw).$' -e '.*/cookies\.txt.$|.*/~.* '
    108 }}}
    109 That should all be on one line. 
    111 Notes:
    112  * You have to flop the directory delimiters (from \ to / ) and remove the extra \ that was used as a directory-character escape (but NOT the other escapes, like \. ), because bbackupquery output flops the delimiters (a kind of bug, I guess). 
    113  * For perhaps related reasons, we have to put a . before the line-terminator $ (it only took me about an hour to figure that out, and, btw, another hour to figure out to use egrep and not grep). 
    114  * I salted(?) my test regex (NOT my bbackupd.conf regex) with a regex of a file that I know is in the store on the server (.qbw files for !QuickBooks), just to make sure that the test is working. 
    115  * You can combine your !BackupLocation regex with multiple egrep -e flags (egrep -e 'regex1' -e 'regex2'). 
    116  * You can probably remove the leading .* from the regex, in this case (.*/cookies\.txt is probably the same as /cookies\.txt here). 
    118 Here's my actual run of this test, lightly redacted:
    119 {{{
    120 [email protected] /cygdrive/c/Program Files/Box Backup
    121 $ ./bbackupquery "list -odtsr" quit | egrep -e '\.([aA][vV][iI]|[iI][sS][oO]|[mM][pP][eE]?[3gG]|[tT][mM][pP]|[bB][aAcC][kK]|[dD][bB][kK]|[bB][kK][~!1-9]|[mMtT][bB][kK]|[oO][lL][dD]|[sS][aA][vV]|[sS][wW][pP]|[xX][lL][kK]|[cC][sS][mM]|[dD][sS][kK]|[oO][bB][jJ]|[pP][aA][rR]|[dD][bB][xX]|[dD][lL][lL]|qbw).$' -e '/thumbs\.db.$|/history\.dat.$|/cookies\.txt.$|/~.* '
    122 No random device -- additional seeding of random number generator not performed.
    124 00000004 f--o-- 2003-10-15T20:44:06 00022 DDataFinanceAccounts/Archive Copy 10-15-2003 Rl.qbw
    125 0000009e f-X--- 2003-10-14T22:02:57 00687 DDataFinanceAccounts/archive/Backup 101503/T4.qbw
    126 00007166 f--o-- 2006-05-09T19:29:04 00454 DDataFinanceAccounts/Hs.qbw
    127 000071c1 f--o-- 2006-05-10T20:20:42 00236 DDataFinanceAccounts/Hs.qbw
    128 000071ec f--o-- 2006-05-11T18:00:11 00224 DDataFinanceAccounts/Hs.qbw
    129 00007215 f--o-a 2006-05-12T20:47:41 28446 DDataFinanceAccounts/Hs.qbw
    130 00007244 f----- 2003-10-15T15:44:06 00876 DDataFinanceAccounts/Archive Copy 10-15-2003 Rl.qbw
    131 0000b397 f--o-a 2006-05-16T16:06:54 26153 DDataFinanceAccounts/Hs.qbw
    132 0000b879 f--o-a 2006-05-17T15:59:52 00370 DDataFinanceAccounts/Hs.qbw
    133 0000b898 f--o-a 2006-05-18T16:36:35 25896 DDataFinanceAccounts/Hs.qbw
    134 0000b8c3 f----- 2006-05-19T17:06:22 39868 DDataFinanceAccounts/Hs.qbw
    135 00004796 -d---- 1969-12-31T19:00:00 00000 DHome/<user>/My Documents/PVSW/Bin/btcompat.bck
    137 [email protected] /cygdrive/c/Program Files/Box Backup
    138 $
    139 }}}
    140 Note that the .bck "file" should NOT show up because .bck is in !BackupLocations !ExcludeFilesRegex ([bB][aAcC][kK]), but notice that it has the "d" flag.  So, that "file" is in fact a directory, and since [bB][aAcC][kK] appears in the section of my !BackupLocations !ExcludeFilesRegex that is intended to trap filename extensions that end the path/filename.ext names, that directory is NOT excluded by my !BackupLocations !ExcludeFilesRegex, but, interestingly, the bbackupquery output for that "file" appears to my test regex to be a "file" that was supposed to be excluded.  A trailing slash in the bbackupquery output (like so: DHome/<user>/My Documents/PVSW/Bin/btcompat.bck/ ) would have prevented this false positive.  All is good. 
    142 (Note that there is a bug somewhere that causes that directory's datestamp to appear as 1969-12-31T19:00:00 when it should be 2005-03-17T11:45:15.)