This is a bit dated.  See readme.txt for changes.

11-Dec-99
Validate.txt

Validate 1.22  Copyright (C)2000 Cornel Huth  All rights reserved [Dec 11 1999]
[Bullet 3.10][WNT][Beta [03:10:09]] (http://40th.com/ -> 40th Floor)
Use: [C:\]validate [-switch] anyfile.dbf [> redirect]

  -all      -cd -mv -mcd -mco
  -sa       show all records (otherwise first & last 5; or -s0 to show none)
  -ss       show scoreboard
  -zc:n     character to display for zero bytes, n=1-255 (def=126)
  -cd       count deleted records
  -mv       memo validate
  -mcd      memo count deleted
  -mco      memo count orphans
  -mhs:n    memo header size (def=blocksize)
  -miu:x    memo in-use hex value (def=0008FFFF)
  -mdump    memo dump block info
  -bu:a     backup anyfile.dbf(.dbt) to a(.dbt)
  -pack     remove all '*' records and their memos
  -sort:a   sort by a [eg, -sort:DESCEND(SALARY)]
  -key:a    crypto key    (-ktf  keep tmp files)
  -h        help (for updates see http://40th.com/)
            note: -sort & -pack invalidate any current index files

 eg: validate any.dbf -bu:save.dbf -pack -sort:ZIP -all -sa -key:secret > x.txt


Validate checks over a DBF/DBT pair and pretty much tells you everything
that could be told on their validity.  It doesn't care about index files
since those contain nothing important except their headers, which are
easy to copy/safeguard (ie, you can recreate an index file at any time;
DBF (records) and DBT (memos) are priceless).  I don't know of any app
that comes close to Validate in its thoroughness.  There's not much to
running it.  Here's a quick lecture:

The DBF can be any valid xBase data file, any size (current builds are
limited to 2GB per file).

The DBT can be any valid xBase memo file, version 4 or later (the kind
with reusable memo space).

Validate also handles all extended field types and file formats that
the Bullet 3.1 database engine handles (lots).



You've got a DBF/DBT (DBT is optional, but most DBFs have DBTs), but
you think it might be corrupt, somewhere.  There's really no way to
know for sure without using Validate.

First thing to do is make a backup:

 validate any.dbf -bu:save.dbf

Bullet doesn't request write access unless -pack or -sort: is used,
but it only take a second or two to make a backup, so why not.  Once
you know that you've got a good, safe backup, start exploring:


 validate any.dbf

This just gets you a quick check of the DBF file, showing the header,
field descriptors, and the first and last five records.  If you have
a lot of records, turn on the scoreboard (-ss) to display the progress.
If the record is longer than the screen width, redirect output to a
file and check it out with a text editor (validate any.dbf > file.txt).
You can check out all records by using -sa, or none with -s0 (ess-zero).
Use -cd to count records marked for deletion.  xBase doesn't remove
data records until a pack is done, instead it just marks them "to be
deleted".


If you have a memo file, check that:

 validate any.dbf -all

-all is a shortcut which includes -cd -mv -mcd -mco.  This does a lot
of checking, back and forth (DBF<->DBT).  If you have a system with a
good disk cache (linux, nt), things should just fly by (use -ss to watch
the progress).


If you want to pack the DBF use -pack.  This removes all deleted
records from the DBF (always have a backup first!).  If a DBT is
attached, it too will be packed right along with the DBF.

 validate any.dbf -pack


If you want to reorder the DBF use -sort:.  You can sort on any field
or combination.  Validate functions are DESCEND(, SUBSTR( and UPPER(.
For example:

 validate any.dbf "-sort:DESCEND(SALARY)+SUBSTR(LNAME,1,4)"

reorders the DBF file so that the first record has the highest salary
(and that in last name order).  See Bullet 3.1 for more on what you
can do.  If you use -sort:, any index files you have will have to be
reindexed.  Or, you can operate on a copy of your data file -- then
you can leave the original alone.




If any problems are detected a notice is displayed.  Here is an
almost-complete list:

 [Alignment Check]

This will only be seen with Bullet 3 extended field types (binary ones
that are not aligned to their natural size -- see Bullet 3.1 for more).
It's there to tell you that you may have problems with RISC CPUs.  The
Bullet 3.1 database engine tells you this at runtime, as well, for both
data record field alignment and key constructs.

 xact pending: ...  [incomplete file shutdown]

Bullet tracks if a given DBF, DBT, IX4 was properly closed.  If not,
you'll see this (separate for DBT and DBF and IX4).

 Not enough memory to check orphans (bytes needed=...

If you are checking for orphaned memo records, Validate sets up a bitmap
in main memory, one bit per memo record.  Not a likely problem.

 *** WARNING: memo file cannot be opened

The DBF file has its first byte (fileID) bits 3 and 7 set to 1, which
indicates that the DBF file has a like-named DBT file, but Validate
didn't find it (DBF/DBT pairs must reside in the same directory).

 *** memo block ... already in another delete chain

This means you've got a corrupt deleted memo chain.  A pack will
probably fix it.  (A pack will probably fix anything.)

 *** ERROR: circular chain in avail list

This means you've got a corrupt deleted memo chain.  A pack will
probably fix it.


 *** WARNING: possible infinite loop in memo avail link, blocks counted=...

This is similar to the circular chain above, but detected in another
DBT test.  A pack will probably fix it.


 *** recNo .../memo field ... is referencing deleted memoNo ... for ... bytes

The DBF record number indicated/memo field indicated is pointing
to a memo allocation that is marked as deleted (the size of the
deleted memo is given).  See next.

 *** recNo=..., memoSlot=... is referencing deleted memoNo=...

This is similar to the above but also accounts for memo blocks that are
part of a larger deletion.  In other words, if a deleted memo is 5 blocks
long, this notice is given if a DBF memo field points to any of the
five blocks.  The previous message is given only when the first block is
pointed to, though the two will probably be seen together (unless a
block 'looks' like a valid memo [has the inUseFlag as its first 4 bytes]
then the previous one won't notice a problem).


 *** WARNING: memoNo=... not ref'ed by DBF and not in deleted list (orphan)

This indicates that a memo block (a memo block is typically a 512-byte
block) is not owned by any DBF record and it is not in the chain of
deleted memos. It is an orphan, forever lost.  A pack will fix this.


 *** NOTE: deleted block# ... is fractional size (... bytes)
 *** NOTE: further notice of fractional size will not be printed
 *** ERROR: DBT is probably a B3 memo file ... Use -or to override
 *** ERROR: DBT might be B2 memo file ... Use -b2 or -or switch

These are related to Bullet 2.x memos.  If you aren't using a Bullet 2.x
memo file and get "might be B2 memo file" then you've got a bad DBT.
A pack will probably fix it.


Here's an example run on a DBF/DBT pair that's been through several memo
updates (adds, expands, shrinks):

Validate 1.2   Copyright (C)1999 Cornel Huth  All rights reserved [Sep 11 1999]
[Bullet 3.10][WNT][Beta [03:10:03]] (http://40th.com/ -> 40th Floor)
cmdline: $demo_01.dbf -cd -mv -mcd -mco -mdump -zc:126

 --------------------------------- DBF detail ---------------------------------
     filename: $demo_01.dbf (local)
       fileID: 0x8B
  last update: 19990911 (yyyymmdd)
      records: 3        (calculated: 3.0)
 deleted recs: 0
 header bytes: 193
 start offset: 193
 record bytes: 64
 xact pending: 0
    encrypted: 0
   file bytes: 385  (0.3 KB)  (0.00 MB)
    last byte: 0x1A

 fld#   fieldname  T  len.dc  offset   notes
 ----  ----------  -  ------  ------  -----------------------------------------
    .  delete tag  *    1.0        0
    1  EMPID~~~~~  C    9.0        1   FTYPE_C:   character, text (0T or not)
    2  LAST_NAME~  C   18.0       10   FTYPE_C:   character, text (0T or not)
    3  FIRST_NAME  C   18.0       28   FTYPE_C:   character, text (0T or not)
    4  SALARY~~~~  N    8.2       46   FTYPE_N:   numeric, text
    5  PERSONAL~~  M   10.0       54   FTYPE_M:   memo field, text# (no 0T)

  Rec# |t|EMPID    |LAST_NAME         |FIRST_NAME        |SALARY  |PERSONAL  |
      1| |10D63AF1~|lastname...~~~~~~~|firstname...~~~~~~|12345.78|0000000004|
      2| |60B7ACD9~|lastname...~~~~~~~|firstname...~~~~~~|12345.78|0000000006|
      3| |3AB50C2A~|lastname...~~~~~~~|firstname...~~~~~~|12345.78|0000000001|

 ================================== DBT dump ==================================
   Blk#     Fileoff  AvailLnk    Blocks   Bytes   ---------- Comments ---------
 00000000  00000000  00000002        1      512   header block
 00000001  00000200  0008FFFF        1       58
 00000002  00000400  00000007        1      504   del
 00000003  00000600  00000008        1      504   del
 00000004  00000800  0008FFFF        1       58
 00000005  00000A00  00000003        1      504   del
 00000006  00000C00  0008FFFF        1       58
 00000007  00000E00  00000005        1      504   del
        inUseFlag is 0008FFFF

 ================================= DBT detail =================================
  8-dot owner: $demo_01
   hdr blocks: 1
  data blocks: 7        (calculated: 7.0)
 max del size: 512      (memo block#=2 [0x2])
   new block#: 8        (0x8)
 deleted blks: 4
 deleted chns: 4
  blocks used: 3        (calculated)
  blocks used: 3        (actual)
  blocks lost: 0        (orphaned)
  bytes/block: 512
 xact pending: 0
   file bytes: 4096  (4.0 KB)  (0.00 MB)

That's it.

This release is a beta, and expires sometime soon.  If you want an update,
you can always get one at:

        http://40th.com/

Follow the path to the latest Bullet database engine.  It's just a
download away.

<eof>
