 |
KiXforms
The Forum for the KiXforms Community
|
|
| View previous topic :: View next topic |
| Author |
Message |
gbarnas
KiXforms Regular
Joined: 07 Mar 2003
Posts: 37
Location: Mahwah, NJ
|
 Posted: Sun Jan 11, 2004 1:07 am Post subject: Building large projects |
 |
|
I'm working on a large KixForms project (currently 200K source). These projects can get pretty unwieldly to manage, so I have some code and processes to help me manage it.
This isn't a KixForms script, but it works especially well for building KixForms projects, so I'm posting it here. I'll post the large project I mentioned above shortly, once testing is complete.
Basically, I have a KixDev folder on my PC. In this folder, I have a KixLib subfolder and several project subfolders. The KixLib folder contains all of my general purpose KIX functions - one or two functions per file. (for personal sanity, I name the ones I write with .KXF extensions, and the ones I download with .UDF extensions, but that's up to you.)
Now, when I start a project, I create a project folder. The main project - usually just a header and comments, is in an appropriatly named .TXT file. Project specific function files, along with files that represent each major collection of forms declarations. I do take care to prefix my file names with FRM## and PRG## so the files are loaded in a sequence I can control - forms first, then functions. (the project TXT file is always first)
When I've completed my collection of individual project files, I run KGEN project - specifying the name of the project TXT file. This causes the following actions to occur:
The name of all functions defined in the files in the KixLib folder are identified.
The project file and all of the .UDF files in the project folder are then scanned for external functions.
Those function files are then scanned to resolve dependencies.
An output file is generated from all the (UDF) files in the project folder.
All required external library files are added to the generated file.
Optionally, comments are stripped from the generated file to reduce its size.
Using this process, my large project is made up of about 30 small files - most are only 2-3K each.
For large projects, I define a BUILD.INI file that has one section and 4 entries, similar to the one below:
| Code: |
[PROJECT]
#Name of project.txt file
Base=sit
# Extension of generated file (.KXW = kixforms script)
Extn=.kxw
# copy to alternate files or directories
Dirs=.\smit.kxw
# strips comments if defined - retains .GEN file with comments
Strip=1
|
Here's the KGEN script itself:
| Code: |
;; kgen - generate a kix script, automatically locating and including required UDFs
;; Glenn Barnas - FRIT/EROC (gbarnas@yahoo.com)
;;
;; Version 1.1 - May 21, 2000
;; Fix - Jan 07, 2004 - Updated to include ALL local project files, even
;; if they don't contain functions (form defs, etc)
;;
;; Usage: kgen project_name
;; Locates all UDF files in the current and library folder & identifies actual
;; udf name, creating a function to file map. Once this is complete, the project
;; file is scanned for any of these functions, and those functions are then scanned
;; for other UDFs. The resulting list of UDF names and files is then sorted and
;; duplicate filenames are removed. This final list of required UDFs is then combined
;; with the source project file in a new output file. Note that the scanning of UDFs
;; for other UDFs IS RECURSIVE!
;;
;; The source file is standard KIX code and can freely reference any UDF contained in
;; the current directory or the common library (defined here as ..\KixLib). Large
;; projects can be broken into smaller pieces and stored in UDFs in the current
;; directory. UDFs that are more generic are stored in a common library. The UDF
;; search path can be customized by adding additional paths to the PATHS variable.
;; The source file must use a standard .TXT extension to avoid confusion with the
;; script output file (.KIX) and project-specific UDFs.
;;
;; The command searches for all files in the library locations that end in "f". This
;; is done because our organization uses .KXF for KiX Function libraries in production
;; environments, and I use .UDF on the dev library to distinguish between "internally"
;; and "externally" developed functions.
;;
;; The kix development structure that this tool supports is as follows:
;; KixDev
;; +--KixLib
;; +--Project1
;; +--Project2
;; +--Projectn
;;
;; From a project folder, run KGEN ProjectName to "compile" the UDFs into a single script.
;;
;; Note that this tool does not detect functions that are called but not defined
;; either within the source file or in an external UDF!
;;
;; The base project name, output file extension, and list of additional directories to
;; copy the project file to can be specified in a BUILD.INI file in the project folder.
;; This allows KGEN to be run with no parameters. There is one section - [Project] - with
;; three keys - Base, Extn, and Dirs. Base is the basename of the project file; Extn is
;; the extension of the generated script (default=.kix); and Dirs is a ';' separated list
;; of paths that the generated script should be copied to. This is a convenient way to copy
;; the script from the project folder to the test environment automatically.
;;
;; KGen has the ability to strip comments from code to reduce the script size (or obfuscate
;; the process?) by specifying a second argument (anything) on the kgen command line.
;; This can also be accomplished via the build.ini file with a STRIP=1 directive.
;; EXTREME CARE SHOULD BE EXERCISED WHEN CHOOSING THIS OPTION! If a semi-colon appears within
;; a string, that line MUST have a comment character at the end of the line or the line will
;; be corrupted! For example: ( $X = ";" ; ) is OK, but ( $X = ";" ) is not.
Break On
Dim $Functions[250,2]
Dim $FnFiles[100]
Dim $File, $Tmp, $P, $FP, $FC, $OP, $FnList, $Ext, $OutDirs, $Dir, $CF, $Parsed, $LoadType
Dim $SrcFile, $OutFile, $LogFile, $CMsg, $Strip, $Line
; DEFAULT OPERATIONAL VARIABLES ARE DEFINED HERE
; list of directories to scan for UDFs - "." is REQUIRED
If %KixIncludePath% <> ""
$PATHS = ".", "..\KixLib", %KixIncludePath%
Else
$PATHS = ".", "..\KixLib"
EndIf
; Should UDFs outside of the project folder be Embedded or Referenced via Call statement?
$LoadType = 0 ; 0=Embedded, 1=Referenced
; Default extension for generated script (usually .KIX for text and .KXW for KixForms)
$Ext = '.kix' ; default output extension
$FP = 0
$Parsed = '' ; list of files parsed
; If BUILD.INI exists, get the base file name and output extension values.
If Exist('build.ini')
If $A1 = ''
$A1 = ReadProfileString('.\build.ini', 'Project', 'Base')
EndIf
If $A2 = ''
$A2 = ReadProfileString('.\build.ini', 'Project', 'Strip')
EndIf
$Tmp = ReadProfileString('.\build.ini', 'Project', 'Extn')
; If output extension is defined, override default of .KIX
If $Tmp <> '' $Ext = $Tmp EndIf
$OutDirs = ReadProfileString('.\build.ini', 'Project', 'Dirs')
EndIf
$SrcFile = $A1 + '.txt' ; project source file name
$OutFile = $A1 + '.gen' ; Generated output file name
$ScrFile = $A1 + $Ext ; project output file name
$LogFile = $A1 + '.log' ; project log file name
; make sure that a name was specified
If $SrcFile = ''
? 'KGen: build file not specified'
? 'KGen: Aborting!' ? ?
Quit
EndIf
; Make sure the specified build file exists
If Not Exist($SrcFile)
? 'KGen: Source file ' + $SrcFile + ' was not found in the current directory'
? 'KGen: Aborting!' ? ?
Quit
EndIF
; Start by deleting the output files
Del $OutFile
Del $LogFile
; Assemble a 2-dimensional array containing a list of function names and the files
; that hold them. Search the folders defined in the PATHS array
'Generating' + Chr(13)
For Each $PATH in $PATHS
; Look for files that end in "F" (.KXF and .UDF)
$File = Dir($Path + '\*.??F')
While $File <> '' And @ERROR = 0
; open the file, read lines until 'function' is found in a non-comment area
; obtain the function name, and add the function name and file name to the array
If Open(3, $Path + '\' + $File,2) = 0
$Line = ReadLine(3)
$Tag = 1
While @ERROR = 0 And $Tag = 1
$Line = Split($Line, ';', 1)[0]
$P = InStr(Trim($Line), 'Function')
; Function definition must be in column 1 after trimming spaces to be considered valid
If $P = 1
; get rid of the 'function ' and split off the function name
$Line = Split(Right($Line, Len($Line) - 9), '(', 1)[0]
$Functions[$FP,0] = $Line
$Functions[$FP,1] = $PATH + '\' + $File
$FP = $FP + 1
$Tag = 0
EndIf
$Line = ReadLine(3)
Loop ; while not error
; Tag will be 1 if no functions are defined in the file. If the .KXF/.UDF file
; is in the project folder (.\) it should be included anyway!
If $Tag = 1 And $Path = '.'
$Functions[$FP,0] = '_FaKe_'
$Functions[$FP,1] = $PATH + '\' + $File
$FP = $FP + 1
EndIf
EndIf
$ = Close(3)
$File = Dir()
Loop
Next ; $Path
; We now have a list of available functions in the library, and the files that contain them.
; The base file specified must be scanned to identify which of these functions are referenced,
; and then those functions must be scanned to determine if any dependencies to other functions
; exist.
$FP = $FP - 1 ; reset the upper limit
$OP = 0
$FnList = ''
; "preload" the function list with all UDF files from the project folder (.)
; These UDF files are always embedded with the base file. They are loaded in
; alphabetical order, so start with numbers to control the sequence.
Gosub PreLoadFn
; Now check the files to determine which external UDFs must be included
; Open the source file and parse it for functions
$InFile = $SrcFile
Gosub FindFn
; Check each of the required function files for dependencies on other functions
; RECURSION IS SUPPORTED!
:Scan
$CF = 0
For Each $InFile in $FnFiles
Gosub FindFn
Next
; If the change flag is set, process the newly added UDFs for additional UDF references
If $CF = 1 Goto 'Scan' EndIf
' ' + Chr(13) ; clear the message line
If $OP = 0
"No UDFs needed for this generation!" ?
Goto 'BUILD'
EndIF
; reduce the array size
ReDim Preserve $FnFiles[$OP - 1]
; sort the array
$FnFiles = QSort($FnFiles)
; remove duplicates
$FnFiles = Uniq($FnFiles)
; If $LoadType <> 0, the referenced external UDF files must be "call"ed first
; this is not yet supported, awaiting a reasonable, universal method
; One way is to start the script with a "gosub _Load_External_UDFS_", and then
; surround the generated call statements with the label and a RETURN statement.
; This isn't "elegant", but putting all the external call statements at the top
; of the generated file may not be pretty either.
; Assemble the output file ($OutFile) from the base and all necessary includes.
:BUILD
$ = RedirectOutput($LogFile)
? 'Building $OutFile from:' ?
; start with the project source file
$CMsg = ';; KixGenerated: @DATE - @TIME @CRLF'
$File = '.\' + $SrcFile
Gosub GenFile
$CMsg = ''
; Add all of the include files
For Each $File in $FnFiles
Gosub GenFile
Next
; Now have a .GEN file - possibly strip comments and generate script file
$Strip = IIf($A2 <> '', 1, 0)
; Delete the script file
Del $ScrFile
$ = Open(4, $OutFile, 2)
If @ERROR = 0
$ = ReDirectOutput($ScrFile)
$Line = ReadLine(4)
While @ERROR = 0
$Comment = InStr(Trim($Line), ';')
If $Strip And $Comment
If $Comment = 1
; skip comment line
Else ; could attempt to remove comments from right-end of line
;$Line ; (or not!)
$Comment = InStrRev($Line, ';') - 1
RTrim(Left($Line, $Comment)) ?
EndIf
Else
$Line ?
EndIf
$Line = ReadLine(4)
Loop
$ = Close(4)
$ = ReDirectOutput('')
EndIf
; Generation is complete - copy to secondary folders if defined
If $OutDirs <> ''
$OutDirs = Split($OutDirs, ';')
For Each $Dir in $OutDirs
Copy $ScrFile $Dir
Next
EndIf
'Generation of $ScrFile is complete!' ? ?
Exit
; SubRoutines ==========================================
; subs use global vars for simplicity in maintaining state
:FindFn
If InStr($Parsed, $InFile) > 0 Return EndIf
If Open(3, $InFile,2) = 0
'Examining $InFile ' + Chr(13)
$ = RedirectOutput($LogFile)
'Examining $InFile ' ?
$Line = ReadLine(3)
While @ERROR = 0
; Get the non-comment line (portion)
$Line = Split($Line, ';', 1)[0]
For $FC = 0 to $FP
$Fn1 = $Functions[$FC,0] + '('
$Fn2 = $Functions[$FC,0] + ' ('
If InStr($Line, $Fn1) > 0 Or InStr($Line, $Fn2) > 0
If InStr($FnList, $Functions[$FC,0]) = 0
$FnList = $FnList + $Functions[$FC,0]
$FnFiles[$OP] = $Functions[$FC,1]
' adding function ' + $Functions[$FC,0] + ' from ' + $Functions[$FC,1] ?
$OP = $OP + 1
$CF = 1 ; set ChangeFlag to force rescan
EndIf
EndIf ; instr fn
Next ; $FC
$Line = ReadLine(3)
Loop
EndIf ; open
$ = Close(3)
$ = RedirectOutput('')
$Parsed = $Parsed + $InFile
Return
; Insure that all of the UDFs in the project directory are loaded, even if they haven't been referenced
; This allows KGEN to generate larger library files
:PreLoadFn
$ = RedirectOutput($LogFile)
'Loading project UDFs' ?
For $Fc = 0 to $FP
If Left($Functions[$FC,1],2) = '.\'
$ = RedirectOutput($LogFile)
$FnList = $FnList + $Functions[$FC,0]
$FnFiles[$OP] = $Functions[$FC,1]
$OP = $OP + 1
' adding function ' + $Functions[$FC,0] + ' from ' + $Functions[$FC,1] ?
EndIf
Next
$ = RedirectOutput('')
Return
; add the named file to the generated script file & update the log
:GenFile
; send output to the destination file
$ = RedirectOutput($OutFile)
If Left($File,2) = '.\' Or $LoadType = 0
$CMsg
Display($File)
; send output to the log file to describe the build
$ = RedirectOutput($LogFile)
' $File' ?
Else
? "Call '$File'" ?
; send output to the log file to describe the build
$ = RedirectOutput($LogFile)
' Call $File' ?
EndIf
; Output back to the screen
$ = RedirectOutput('')
Return
;;
;;======================================================================
;;
;;FUNCTION qsort() (originally "qs()")
;;
;;ACTION Sorts a 1-dimension array
;;
;;AUTHOR "BrianTX"
;;
;;SYNTAX qsort(array)
;;
;;PARAMETERS array - array to be sorted
;;
;;REMARKS Sorts numeric or text (ASCII order) arrays
;;
;;RETURNS array, sorted
;;
;;DEPENDENCIES none
;;
;;TESTED WITH NT4, W2K, WXP
;;
;;EXAMPLES $Sorted = qs($Unsorted)
;
Function qsort($a)
DIM $ls[32],$us[32],$sp,$L,$U,$m,$p,$i,$j,$t
$ls[0] = 0 ; Lower Stack Index
$us[0] = UBOUND($a) ; Upper Stack Index
$sp = 0 ; Stack Pointer
While $sp >=0
$l=$ls[$sp]
$u=$us[$sp]
While $L < $U
$p=$L+($U-$L)/2
$t=$a[$L]
$A[$L]=$A[$P]
$A[$P]=$t
$i=$L+1
$j=$U
:L1
While ($i<$j) AND $A[$L] > $A[$i]
$i=$i+1
Loop
While ($j>=$i) AND $A[$j] > $A[$L]
$j=$j-1
Loop
IF $i >= $j goto L2 ENDIF
$t=$A[$i]
$A[$i]=$A[$j]
$A[$j]=$t
$j=$j-1
$i=$i+1
Goto L1
:L2
$t=$a[$l]
$a[$l]=$a[$j]
$a[$j]=$t
$m=$j
If $m-$l <= $u - $m
If $m+1 < $u
$ls[$sp]=$m+1
$us[$sp]=$u
$sp=$sp+1
Endif
$u=$m-1
Else
If $m-1 > $l
$ls[$sp]=$l
$us[$sp]=$m-1
$sp=$sp+1
Endif
$l=$m+1
Endif
Loop
$sp=$sp-1
Loop
$qsort=$a
Endfunction
;;
;;======================================================================
;;
;;FUNCTION uniq()
;;
;;ACTION Removes duplicates from a sorted, 1-dimension array
;;
;;AUTHOR Glenn Barnas / FRIT-EROC
;;
;;SYNTAX uniq(array)
;;
;;PARAMETERS array - array to remove duplicates from
;;
;;REMARKS Array must be sorted first - no check for sort is performed
;;
;;RETURNS array with duplicate items removed
;;
;;DEPENDENCIES none
;;
;;TESTED WITH NT4, W2K, WXP
;;
;;EXAMPLES $Singles = Uniq($has_dups)
;
Function Uniq($A)
; $TOP is last entry, $X is source pointer, $Y is destination pointer
Dim $Top, $X, $Y
$Top = UBound($A) ; Find size of original array
Dim $WA[$Top] ; Create working array
$Y = 1 ; output array pointer
$WA[0] = CStr($A[0]) ; copy first element
For $X = 1 to $Top
If $WA[$Y - 1] <> $A[$X] ; is current value different from last?
$WA[$Y] = CStr($A[$X]) ; add it to output
$Y = $Y + 1 ; increment output pointer
EndIf
Next
ReDim Preserve $WA[$Y - 1] ; resize output array
$uniq = $WA ; return the array of unique elements
EndFunction
|
I use a simple KGEN.BAT file to invoke this, passing the arguments to the kix script:
@Kix32.exe kgen.kix $A1="%1" $A2="%2"
OK - teaser! Here's a screenshot of the large (13,000+ lines) project I'm working on...
 |
|
| Back to top |
|
 |
gbarnas
KiXforms Regular
Joined: 07 Mar 2003
Posts: 37
Location: Mahwah, NJ
|
| Posted: Wed Dec 02, 2009 1:19 am Post subject: KixDev - KGen+Sanity |
|
|
This project has evolved into the KixDev package, which is built around KGen. KGen resolves UDF dependencies and combines multiple package and UDF files into a finished script. It can then optionally tokenize the resulting script, and copy it to other locations.
From a developer standpoint, it generates several reports on variable usage (declaration, first use, undeclared, declared but unused) and code structure. It performs a sanity check on the resulting script, looking for mismatched quotes, parenthesis, and paired functions such as If/Endif and While/Loop.
We use KGen internally for all of our Kix and KF script development.
Version 2.4 is available for download from my web site - www.innotechcg.com - in the Products / Admin Toolchest section. This is just one of the free tools we offer to the Kix development community. The ZIP package includes the KGen script, a PDF User Guide, and a copy of our entire development UDF library.
Glenn |
|
| Back to top |
|
|
|
|
You cannot post new topics in this forum
You cannot reply to topics in this forum
You cannot edit your posts in this forum
You cannot delete your posts in this forum
You cannot vote in polls in this forum
You cannot attach files in this forum
You can download files in this forum
|
Powered by phpBB © 2001, 2005 phpBB Group
|
FAQ
Search
Memberlist
Usergroups
Register
Profile
Log in to check your private messages
Log in
