PBS (Perl Build System) pronounced "peps".
Cons Mailing list
itchiness and irritations
Private communications
Daniel Pfeiffer:
http://dapfy.bei.t-online.de/make.pl/
Nadim khemir. Teleca Mobile Technology.
Anders Lindgren. Anoto.
Ola Martensson. Cybercom.
Problems at hand
Fundings
Sudden stop
Has been around for 20 years
Found most everywhere
Free
Fast and small
Compact rule syntax
Rule based
Nice hair cut
Nice beard
Nice tie
Evenly sized ears
Sympathetics look
Not for complex systems
http://www.canb.auug.org.au/~millerp/rmch/recu-make-cons-harm.html
Cryptic syntax
Crippled script language
Unclear (too smart) evaluation points
Monolithic and notoriously difficult to maintain
Needs an on site guru
Not (easily) found on windows, which is the main platform used by developers (like it or not)
Various slightly incompatible version of 'make'
Some built-in rules which are best disabled (-r option) (from Xmake documentation)
Will not automatically detect source files, dependencies or create rules unless you write the makefiles to do these things. (from Xmake documentation)
Is make broken?
Whether it's because of shortcomings in "gmake", or simply because people want to try something new, lots of other build systems exists
Ant
http://www.theserverside.com/news/thread.jsp?thread_id=23426
Cook
Cons
Cmake
Scons
MakePP
Make.pl
Brazil
Jam
Odin
SCons
Otto
Jake
Nmake
Xmake
...
http://www.cbel.com/build_management/
http://www.linuxlinks.com/Software/Programming/Development/Tools/Make_Tools/
CC = gcc CFLAGS = -Wall \ -DG_DISABLE_DEPRECATED \ -DGDK_DISABLE_DEPRECATED \ -DGDK_PIXBUF_DISABLE_DEPRECATED \ -DGTK_DISABLE_DEPRECATED ttt_test: tictactoe.o ttt_test.o $(CC) ttt_test.o tictactoe.o -o ttt_test `pkg-config --libs gtk+-2.0` ttt_test.o: ttt_test.c tictactoe.h $(CC) -c ttt_test.c -o ttt_test.o $(CFLAGS) `pkg-config gtk+-2.0 --cflags` tictactoe.o: tictactoe.c tictactoe.h $(CC) -c tictactoe.c -o tictactoe.o $(CFLAGS) `pkg-config gtk+-2.0 --cflags` clean: rm -f *.o ttt_test
Low level access
Detail over exposure
Simple for simple tasks
Often developed and maintained by the same person
Import qw( CONS BIN ); InstallAs $CONS "$BIN/byacc_Fs.exe", 'byacc.exe' ; $CONS->Program ( 'byacc.exe' ,qw ( closure.c error.c lalr.c lr0.c main.c mkpar.c output.c reader.c skeleton.c symtab.c verbose.c warshall.c ) ) ;
Let you think at a higher level
Hide the mundane details
Sell "their" way of doing things
Not easy to get into the details
http://www.dsmit.com/cons/stable/cons.html
More and more systems use scripted languages, Ruby, Tcl, Python and Perl.
Many of the programs listed above use Perl.
make.pl (RIP)
Makepp
One or two modules on CPAN to handle dependencies
PBS
Be readable by a human (no XML)
Be understandable by a human (no gmake or home grown syntax)
Be easy to develop and extend (no C)
Be fun and powerful (Perl)
No 'veryclean' command
Low and High Level
No magic
Go where no one has gone before
Trigger, Documentation, debugging, visualization, user help, ...
PBS - Set of perl modules to implement build utilities.
pbs - one front end to PBS
pbs is a build utility in the same spirit as gmake.
pbs was developed to tackle the build of complex systems.
pbs neither tries to make the task easy nor fast but merely possible.
pbs is written in Perl and uses Perl exclusively for defining the system to build.
pbs has a full featured scripting language, Perl.
pbs only introduces a few new functions.
Simple, expandable, manageable
Not reinvent the wheel
3 passes architecture/Process
Each pass is implemented in a separate module, giving you the ability to depend and check a system without building it or building it 'manually'.
[nadim@khemir warp]$ pbs -tt -tno ** Depending [PBS/0] ** Depending './b1' with sub pbs 'W1:./warp1.pl' No user defined [W1] Build(), using DefaultBuild() with [BuiltIn, User] rules and [BuiltIn, User] configs. ** Depending [W1/1] ** ** Checking ** Tree for __PBS_warp_tree: `- ./all [H1] |- ./a1 [H2] |- ./a2 [H3] |- ./a3 [H4] |- ./b0 [H5] |- ./b1 [H6] | |- ./c1 [H7] | | |- ./b0 [H8 -> H5] | | |- ./d1 [H9] | | |- ./f1.nad [H10] | | |- ./f2.nad [H11] | | |- ./f3.nad [H12] | | |- ./f4.nad [H13] | | `- ./f5.nad [H14] | `- ./c2 [H15] | `- ./d1 [H16 -> H9] |- ./b2 [H17] |- ./b3 [H18] `- ./source [H19] ** Building ** Number of nodes in the dependency tree: 20. 2 [0V] nodes scheduled for build. #------------------------------------------------------------------------------ Node './b1' [/home/nadim/Dev/PerlModules/PerlBuildSystem-0.25/Pbsfiles/warp/out/b1] : touch /home/nadim/Dev/PerlModules/PerlBuildSystem-0.25/Pbsfiles/warp/out/b1 #------------------------------------------------------------------------------ Node './all' [/home/nadim/Dev/PerlModules/PerlBuildSystem-0.25/Pbsfiles/warp/out/all] : touch /home/nadim/Dev/PerlModules/PerlBuildSystem-0.25/Pbsfiles/warp/out/all Build Done.
[nadim@khemir pnw2004]$ pbs -h | wc -l PerlBuildSystem: PBS [-p Pbsfile[.pl]] [[-switch]...] target [target ...] switches: 152
The switches can be categorized as follow:
Setup
Pbsfile
Repository
Defines
Verbosity
etc ..
Debugging
Visualization
PRF
Anonymous prf (Pbs.prf)
User prf
-nh -build_directory ./user_out -sd . -sd $NAILARA all
PBS_FLAGS
PBS_LIB_PATH
Pbsfiles are package 'less' perl scripts. Pbsfile can have any name you operating system supports.
If no Pbsfile is given on the command line, pbs will try:
Pbsfile.pl
pbsfile.pl
pbsfile
If no Pbsfile.pl is found, pbs exits with an error message.
Pbsfiles being perl scripts, they follow the order of evaluation any perl script follows.
code is parsed by perl
modules are loaded and compiled
code is evaluated
After perl evaluation, the build engine takes over
You write Pbfiles as package less scripts but PBS fiddles with you file.
Original:
AddRule [VIRTUAL], 'all', ['all' => 'file.o:1.0', 'file2.o:1.0'], "echo hi" ;
Modified:
#>>>>> start of file '/home/nadim/Dev/PerlModules/PerlBuildSystem-0.25/pnw2004/examples/package_fiddle.pl' #line 0 '/home/nadim/Dev/PerlModules/PerlBuildSystem-0.25/pnw2004/examples/package_fiddle.pl' package PBS::Runs::PBS_1 ; use strict ; use warnings ; use PBS::Constants ; use PBS::Shell ; use PBS::Output ; use PBS::Rules ; use PBS::Triggers ; use PBS::PostBuild ; use PBS::PBSConfig ; use PBS::Config ; use PBS::Check ; use PBS::PBS ; use PBS::Digest; PBS::Digest::AddFileDependencies('PBSFILE:/home/nadim/Dev/PerlModules/PerlBuildSystem-0.25/pnw2004/examples/package_fiddle.pl') ; #line 1 '/home/nadim/Dev/PerlModules/PerlBuildSystem-0.25/pnw2004/examples/package_fiddle.pl' AddRule [VIRTUAL], 'all', ['all' => 'file.o:1.0', 'file2.o:1.0'], "echo hi" ; # load OK 1 ; #<<<<< end of file '/home/nadim/Dev/PerlModules/PerlBuildSystem-0.25/pnw2004/examples/package_fiddle.pl'
makes a package of your Pbsfile
use strict and warnings
use many of the PBS modules
Adds a package dependency
sync the line numbering
returns success
AddRule [VIRTUAL], 'all', ['all' => 'ds_master.pdf'], BuildOk("Done."); AddRule 'tex2pdf', ['*.pdf' => '*.tex'], '/usr/in/tex2pdf %FILE_TO_BUILD %DEPENDENCY_FILES' ; AddRule 'master', ['ds_master.tex' => @tex], \&BuildMaster;
Rule Components:
Type
Rule name
Depender
Builder
Arguments
PBS is, obviously, not compatibility with gmake. And that's good!
If multiple rules match a node/file, the sum of the dependencies returned by matching dependers will become the node/file dependencies.
AddRule 'o_c', ['*.o' => '*.c'] ; AddRule 'o_s', ['*.o' => '*.s'] ;
are used on file compress.o, the dependers would generate the following dependencies: compress.c and compress.s.
gmake is, IMO, too magical in its way of handling your rules. I don't mean it is wrong but that it simply doesn't fit the pbs way of generating dependencies.
Node './nailara/os/rtos/rtos.objects' [/devel/nailara/projects/argus2_test/nadim_out/nailara/os/rtos/rtos.objects] : Inserted at /devel/nailara/os/rtos/Pbsfile.pl [rtos]:__ROOT:PBS::Runs::rtos_1:BuiltIn:PBS_INTERNAL_/usr/local/lib/perl5/site_perl/5.8.0/PBS/PBS.pm:253. dep: ./nailara/os/rtos/rtos.o [/devel/nailara/projects/argus2_test/nadim_out/nailara/os/rtos/rtos.o] ./nailara/os/rtos/irq.o [/devel/nailara/projects/argus2_test/nadim_out/nailara/os/rtos/irq.o] ./nailara/os/rtos/critical.o [/devel/nailara/projects/argus2_test/nadim_out/nailara/os/rtos/critical.o] ./nailara/os/rtos/reset.o [/devel/nailara/projects/argus2_test/nadim_out/nailara/os/rtos/reset.o] ./nailara/os/rtos/device.o [/devel/nailara/projects/argus2_test/nadim_out/nailara/os/rtos/device.o] ./nailara/os/rtos/context.o [/devel/nailara/projects/argus2_test/nadim_out/nailara/os/rtos/context.o] ./nailara/os/rtos/exceptions.o [/devel/nailara/projects/argus2_test/nadim_out/nailara/os/rtos/exceptions.o] ./nailara/os/rtos/irq_asm.o [/devel/nailara/projects/argus2_test/nadim_out/nailara/os/rtos/irq_asm.o] ./nailara/os/rtos/os_init_argus2.o [/devel/nailara/projects/argus2_test/nadim_out/nailara/os/rtos/os_init_argus2.o] ./nailara/os/rtos/exceptions_asm.o [/devel/nailara/projects/argus2_test/nadim_out/nailara/os/rtos/exceptions_asm.o] rebuild because of: __SELF (Doesn't exist) matching rule: #6[B] 'Object file list:PBS::Runs::rtos_1:User:/devel/nailara/Pbs/Rules/BuildSystem.pm:52' => none matching rule: #7 'Object_files_0 @ '/devel/nailara/os/rtos/Pbsfile.pl':17 :PBS::Runs::rtos_1:User:/devel/nailara/Pbs/Rules/BuildSystem.pm:139' => ./nailara/os/rtos/rtos.o ./nailara/os/rtos/irq.o ./nailara/os/rtos/critical.o ./nailara/os/rtos/reset.o ./nailara/os/rtos/device.o ./nailara/os/rtos/context.o ./nailara/os/rtos/exceptions.o ./nailara/os/rtos/irq_asm.o matching rule: #8 'Object_files_1 @ '/devel/nailara/os/rtos/Pbsfile.pl':29 :PBS::Runs::rtos_1:User:/devel/nailara/Pbs/Rules/BuildSystem.pm:139' => ./nailara/os/rtos/os_init_argus2.o matching rule: #9 'Object_files_2 @ '/devel/nailara/os/rtos/Pbsfile.pl':70 :PBS::Runs::rtos_1:User:/devel/nailara/Pbs/Rules/BuildSystem.pm:139' => ./nailara/os/rtos/exceptions_asm.o Using builder: #6 'Object file list:PBS::Runs::rtos_1:User:/devel/nailara/Pbs/Rules/BuildSystem.pm:52'
A depender is a perl sub.
sub Depender { my ($dependent_to_check, $config, $tree, $inserted_nodes, $dependencies, $builder_override, $argument_override) = @_ ; my $build_directory = $tree->{__PBS_CONFIG}{BUILD_DIRECTORY} ; my $source_directories = $tree->{__PBS_CONFIG}{SOURCE_DIRECTORIES} ; my @my_dependencies ; if(defined $dependencies && @$dependencies && $dependencies->[0] == 1 && @$dependencies > 1) { # previous depender defined dependencies unshift @my_dependencies, $dependencies->[1 .. -1] ; } unshift @my_dependencies, 1 ; return(\@my_dependencies, $builder_override, $argument_override) ; }
Dependers are chained
Dependers are powerful
Depender can be very complex
A depender is a perl sub but we can build the sub in different ways.
AddRule 'a.o_rule name', ['a.o' => \&SpecialDepender], ... AddRule 'a.o_rule name', ['a.o' => 'a.c', '/path/file', \&SpecialDepender], ... AddRule 'a.o_rule name', ['a.o' => 'a.c', '/path/file'], ... AddRule 'r4', ['a.h' => '[path]/[basename].[ext]'] ; # this is cyclic of course.
AddRule 'r1', ['*/*.c' => '*.h'] ; AddRule 'r2', ['*/*.c' => 'somefile'] ; AddRule 'r3', ['*/*.c' => '/full_path/somefile'] ;
If the dependent regexp is a regexp object or a sub, PBS (AddRule) considers the rule to be a pure perl rule.
AddRule 'rule_1', [qr<\./all$> => '$path/muu/all', '$path/f1', '$path/a.o', '$path/muu/xxxxx.o'] ; AddRule 'rule_2', [qr<\.o$> => '$path/$basename.c'] ; AddRule 'rule_3', [ # creator [sub{return(@_[4 .. 6])}] => #regexp AndMatch(qr<\.c$>, NoMatch(qr/xx/)) => #dependencies '$path/$basename.h' , [ # post depender sub { return([1, "hi_there2"], @_[5 .. 6]) } ] , sub #depender { return([1, "hi_there1"], @_[5 .. 6]) } ] ;
Available variables:
$path
$basename
$name
$ext
Composite match
[nadim@khemir pnw2004]$ pbs -p ../Pbsfiles/pure_perl_rule/Pbsfile.pl -tta -tt -tno all ... ** Checking ** Tree for __PBS_root_1_pbs_.._Pbsfiles_pure_perl_rule_Pbsfile.pl: `- ./all [H1] |- ./a.o [H2] | `- ./a.c [H3] | `- hi_there2 [H4] |- ./f1 [H5] |- ./muu/all [H6] `- ./muu/xxxxx.o [H7] `- ./muu/xxxxx.c [H8]
Single shell command:
AddRule 'rule_name', ['X' => 'x1', 'x2'], "touch %FILE_TO_BUILD" ;
Perl sub:
AddRule [VIRTUAL], 'rule_name', ['test' => 'all'], sub{ 1, "test OK\n"} ;
Alternatively define your builders in a perl module.
use YourModule ; #or PbsUse 'YourPbsModule' ; AddRule 'rule_name', ['X' => 'x1', 'x2'], \&YourBuilder ;
Multiple commands:
AddRule [VIRTUAL], 'test', ['test' => 'all'], [ "touch %FILE_TO_BUILD", # hmm! sub{ 1, "test OK\n"} ; ]
AddRule 'c_objects', [ '*.o' => '*.c' ], '%CC %CFLAGS -c -o %FILE_TO_BUILD %DEPENDENCY_LIST' ;
AddRule [VIRTUAL], '1', [ 'test' => 'b'], BuildOk('Done', 1) ;
The last argument passed to AddRules is passed as an argument to the builder.
AddRule 'o_c', ['*.o' => '*.c'], \&BuildAnObject ;
It is up to the builder to interpret the argument (a scalar that can point to other perl types if you want it to). The argument to builder allows to do something special on a specific node. For example, you'd like to compile a specific C file with a -O2 switch to the compiler. You could use the following rules:
# General rule to build object file AddRule 'o_c', ['*.o' => '*.c'], \&BuildAnObject ; #specific rule AddRule 'special_o_file', ['special_file.o' => 'special_file.c'], undef, '-O2';
All the .o files will be generated by BuildAnObject builder. When building 'special_file.o', BuildAnObject will be passed the argument '-O2'.
The last defined argument for a node/file is passed to the builder. PBS will warn you if multiple arguments are selected.
PBS having no built-in rules, it would be cumbersome to have to redefine the rules you use in all the Pbsfiles. PBS support in include mechanism that looks like perl's 'use'l. PbsUse takes the name of a file which contains rules or configuration variables definitions.
File '.../Rules/C.pm': AddRule 'exe', [exe => undef], \&BuildAnExe ; AddRule 'O_to_C', ... AddRule 'C_to_H', ... ...
You can then include it in you Pbsfile.
PbsUse('Rules/C') ;
PbsUse will automatically append '.pm' at the end of the file name. If the file can't be found in the same directory as the Pbsfile, the environment variable PBS_LIB_PATH will be used to point to the directories where the files are to be searched.
AddRule 'c_objects', [ '*/*.o' => '*.c' ], ... AddRule 'cpp_objects', [ '*/*.o' => '*.cpp' ], ... AddRule 's_objects', [ '*/*.o' => '*.s' ], ... AddRuleTo 'BuiltIn', [META_RULE], 'o_cs_meta', [\&FirstAndOnlyOneOnDisk, ['cpp_objects', 'c_objects', 's_objects'], 'c_objects'] ;
When you define the above 'o_meta' rule, PBS removes the slave rules from it's rule list (in the current package only). FirstAndOnlyOneOnDisk will be called with a reference to the slaves rules as arguments. This allows you to define your own 'magic'. FirstAndOnlyOneOnDisk source code can be found in the distribution.
Unlike other build systems, PBS doesn't define any Built-in rule. It is very easy to define and use libraries of rules. Those become your Built-in rules.
Why?
PBS team doesn't own a crystal ball
No rule collisions (except yours)
No magic (except yours)
file://home/nadim/Dev/PerlModules/PerlBuildSystem-0.25/PBSLib/Rules/C.pm
In Nailara, Rules/C_depender.pm implements a C file depender.
Depending './nailara/os/rtos/rtos.objects' with sub pbs 'rtos:/devel/nailara/os/rtos/Pbsfile.pl' Generating '/devel/nailara/os/rtos/critical.c' dependency file './nailara/os/rtos/critical.c' Includes: |- /devel/nailara/hal/argus2/argus2.h [H1] | `- /devel/nailara/hal/pagetable.h [H2] | `- /devel/nailara/hal/argus2/pagetable/pagetable.h [H3] |- /devel/nailara/hal/argus2/cp15.h [H4] |- /devel/nailara/hal/argus2/cpu.h [H5] | |- /devel/nailara/hal/argus2/argus2.h [H6 -> H1] | `- /devel/nailara/hal/argus2/cp15.h [H7 -> H4] |- /devel/nailara/hal/argus2/pagetable/pagetable.h [H8 -> H3] |- /devel/nailara/hal/cpu.h [H9] | `- /devel/nailara/hal/argus2/cpu.h [H10 -> H5] |- /devel/nailara/hal/pagetable.h [H11 -> H2] |- /devel/nailara/os/rtos/critical.h [H12] |- /devel/nailara/os/rtos/prios.h [H13] `- /devel/nailara/os/rtos/rtos.h [H14] `- /devel/nailara/os/rtos/prios.h [H15 -> H13]
file://devel/nailara/projects/argus2_test/nadim_out/nailara/os/rtos/critical.c.depend
Remove rules
Replace rules
AddRule 'gnu asm stubs', [ '*/*.o' => '*.stub' ], "%CC -x assembler-with-cpp %CFLAGS %CDEFINES %CFLAGS_INCLUDE -I%PBS_REPOSITORIES -o %FILE_TO_BUILD -c %DEPENDENCY_LIST" ; ReplaceRule [META_RULE], 'o_cs_meta', [\&FirstAndOnlyOneOnDisk, ['c_objects', 'gnu asm stubs'] , 'gnu asm stubs'] ;
create rules at run time
Adding configuration to your Pbsfile:
AddConfig
AddConditionalConfig
ConfigVariableNotDefined
ConfigVariableEmpty
ConfigVariableNotDefinedOrEmpty
Environment variables
Commend line variables
AddConfig 'a' => 1 ; AddConfig 'a' => 2 ; AddConfig 'b:locked' => 1 ; AddConfig 'b' => 2 ;
Gives this when run:
[nadim@khemir pnw2004]$ pbs -p ../Pbsfiles/config/lock.pl -tta all No source directory! Using '/home/nadim/Dev/PerlModules/PerlBuildSystem-0.25/pnw2004'. No Build directory! Using '/home/nadim/Dev/PerlModules/PerlBuildSystem-0.25/pnw2004'. Overriding config 'PBS::Runs::PBS_1::CURRENT::User::a' it is now: |- ORIGIN [A1] | |- 0 [S2] = PBS::Runs::PBS_1:'../Pbsfiles/config/lock.pl':14 => 1 | `- 1 [S3] = PBS::Runs::PBS_1:'../Pbsfiles/config/lock.pl':15 => 2 `- VALUE [S4] = 2 Configuration variable 'b' defined at PBS::Runs::PBS_1:'../Pbsfiles/config/lock.pl':18, wants to override locked variable: PBS::Runs::PBS_1::CURRENT::User::b: |- LOCKED [S1] = 1 |- ORIGIN [A2] | `- 0 [S3] = PBS::Runs::PBS_1:'../Pbsfiles/config/lock.pl':17 => 1 `- VALUE [S4] = 1 Died at /usr/local/lib/perl5/site_perl/5.8.0/PBS/Config.pm line 409. ...propagated at /usr/local/lib/perl5/site_perl/5.8.0/PBS/PBS.pm line 483.
Run Example:
time pbs -p ../Pbsfiles/config/force.pl -dc all
VIRTUAL
LOCAL
FORCED
IMMEDIATE_BUILD
AddRule 'object1',['object.o' => 'object.c:4.2.0'] ;
You can give attributes to the nodes, the attribute is declare by following the node name with a colon and a textual attribute (see the example bellow). When PBS finds such an attribute, it calls a user sub registrated via RegisterUserCheckSub. The user sub receives the following arguments:
The full name for the node (path and name)
The attribute declared in the rule
The sub should return a file full name (path/name) this is most often the first argument it receives, or die with an error message. The node attribute could be used to, for example, verify the version of a node.
RegisterUserCheckSub ( sub { my ($full_name, $user_attribute) = @_ ; #print "$full_name => $user_attribute\n" ; return($_[0]) ; # must return a file name } ) ;
AddRule 'all', [ all => '1', '2', 'cyclic'] ; AddRule 'cyclic', [cyclic => 'cyclic2'] ; AddRule 'cyclic2', [cyclic2 => 'cyclic3'] ; AddRule 'cyclic3', [cyclic3 => 'cyclic'] ;
Example 2:
AddRule 'test1', ['all' => 'HERE', 'A'] ; AddRule 'test2', ['HERE' => 'x.z'] ; AddRule 'test3', ['A' => 'x.z'] ; AddRule 'test4', ['this.lib' => 'HERE', 'lib.z'] ; AddRule 'test5', ['all' => 'lib.h', 'HERE'] ; AddRule 'test6', ['x.z' => 'all'] ; AddRule 'test7', ['lib.h' => 'HERE'] ;
Run example:
time pbs -p ../Pbsfiles/test1/cyclic.pl all and with -o and -dd switch
Only one cycle is show n in the generated graph!
Devel::Cycle is good
Using --source_directory or --sd, you can direct PBS to search for files in other source directories than the current directory.
You can specify multiple --sd switches.
PBS will search the directories in the order you specify them.
example in /devel/nailara/projects/argus2_test try -dsd -dsi -daa
Using --build_directory, you can have PBS place the generated files in a directory different from the current directory. This allows you to separate your source files from the generated files.
If no build or source directory is specified, PBS will use the current directory. If you specify source directories, PBS will search exclusively in the specified directories. The current directory is not searched. If you want the current directory and other directories to be searched, you must specify the current directory too.
When no default build and/or source directory is given, pbs will display an information line.
No source directory! Using '/devel/nailara/projects/argus2_test'. No Build directory! Using '/devel/nailara/projects/argus2_test'. No user defined [PBS] Build(), using DefaultBuild() with [BuiltIn, User] rules and [BuiltIn, User] configs. ** Depending [PBS/0] ** Depending './nailara/nailara.objects' with sub pbs 'NAILARA:/devel/nailara/Pbsfile.pl'
Debugging without rebuilding the world
--mandatory_build_directory and user PRF
Within a pair of matching curly braquets'{}', list:
The node name
The Pbsfile name.
2 The Package name (the sub package in which to run the sub build).
3 Optional. Extra variables you would like to set in the sub depend
AddRule 'sub_depend', { NODE_REGEX => 'x.lib' , PBSFILE => './P2.pl' , PACKAGE => 'LIB' , BUILD_DIRECTORY => '/bd_P2' , SOURCE_DIRECTORIES=> ['/sd_P2_2', '/sd_P2_1'] } ;
Subpbsfiles are Pbsfiles.
No rule Inheritance. Period!
show example with -dur in any nailara project
When PBS starts a sub Pbsfile, PBS pushes (merges) the the parent configuration in the child Pbs. This is done automatically by PBS just before calling the child Build() sub or the default Build() sub.
Magnus Ladulås, hertig, kung av Sverige 1275-1290, född 1240, död 18 december 1290 på Visingsö.
http://sv.wikipedia.org/wiki/Magnus_Ladulås
LOCKED
UNLOCKED
FORCE
OVERRIDE_PARENT
SILENT_OVERRIDE
[nadim@khemir PerlBuildSystem-0.25]$pbs -p Pbsfiles/config/parent.pl -no_build -tt -tno -dc -dpos parent
Overriding a file
Overriding a Pbsfile
/devel/nailara |-- Pbs |-- communication |-- ... |-- hal | |-- argus | |-- argus2 | |-- arguslt | |-- drivers | `-- pcb | |-- arguslt | |-- olympus | |-- olympus2 | | |-- P1 | | `-- bootinfo-headers | | `-- testprograms | `-- olympuslt |-- ... |-- projects | |-- gdb_example | | |-- nadim_out | | | `-- nailara | | | |-- hal | | | | |-- argus | | | | | `-- pagetable | | | | |-- drivers | | | | | |-- nand | | | | | `-- uart | | | | `-- pcb | | | | `-- olympus | | | `-- os | | | |-- debug | | | |-- fs | | | | |-- nandfs | | | | `-- vfs | | | |-- mem | | | | `-- poolheap | | | |-- rtos | | | `-- utils | | `-- nailara | | |-- hal | | | `-- pcb | | | |-- nadim | | | `-- xxolympus | | `-- os | | `-- mem | | `-- poolheap | `-- usbmass_demo `-- utils
(pbs -bi '*Bitmap*')
(./gdb_example/nailara/hal/pcb/xxPbsfile.pl)
PBS automatically generates digests when a node is build. Time Stamps are ignored(*).
The digest contains the following elements:
If all the elements needed to build the node are found in the file's digest, PBS uses it, otherwise the node is rebuild.
PBS will expects a digest for all nodes/files. We have to tell PBS how to make the difference between a generated file and a source file.
ExcludeFromDigestGeneration() allows you to exempt a certain type of files from the digest.
ExcludeFromDigestGeneration('c_files' => qr/\.c$/) ;
The first argument is a description string, the second one a reference to a regexp object. Node names matching the regexp will be exempted.
Some source files are automatically generated (ex by flex, yacc, your own generators, ...), ForceDigestGeneration() can selectively re-impose a digest on a certain file that would have been exempted by ExcludeFromDigestGeneration.
ForceDigestGeneration( 'a.c is generated' => qr/a\.c$/) ;
Example of PBS generated digest:
# This file is automaticaly generated by PBS (Perl Build System). # File: /devel/nailara/projects/argus2_test/nadim_out/main.o.pbs_md5 # Date: Tue Mar 9 20:34:25 2004 # User: nadim @ khemir.net # PBS_LIB_PATH: /home/nadim/Dev/PerlModules/PerlBuildSystem-0.25/PBSLib/:/devel/nailara/Pbs/:/devel/nailara # Pbsfile: ./Pbsfile.pl $digest = { './main.o' => 'dc74f3b18b88e845027a683a68c88cbb', '__NODE_VARIABLE:CDEFINES' => ' -DARCH_OLYMPUS2=1 -DARGUS2=1 -DCPU_ARGUS2=1 -DEXCEPTION_BASE=0x06000000 -DKERNEL=1 -DKHEAP_32BIT_POOL=1 -DKHEAP_SIZE=0x20000 -DPROC_MAX_PROCESSES=16 -DPROC_QUANTUM=1000000 -DROBASE=0x06000000 -DROBASE_PHYS=0x06000000 -DRWBASE=0x06100000 -DRWBASE_PHYS=0x06100000 -DRWLIMIT=0x08000000', '__PBS_LIB_PATH/Builders/HostedBuilder.pm' => '080bac06fe0fc6338ccd64f6c02a75cb', '__PBS_LIB_PATH/projects/argus2_test/Config.pm' => '28dc45bf0943c1a97745de98ba82b8b6', '__PBS_LIB_PATH/Configs/Projects/Nailara.pm' => 'c85daa008b4d5eb2abbbff59abf5d6ca', '__PBS_LIB_PATH/Configs/Compilers/gcc.pm' => '680336b56f1a9614a270114e3a03a6c9', '__PBS_LIB_PATH/Dependers/Locator.pm' => '1695797f4dea0a29cd755606daf7af2b', '__PBS_LIB_PATH/ShellConfig.pm' => '6cc04b44ec134dca90b6eddd13716550', '__PBS_LIB_PATH/Rules/Nailara.pm' => 'ee2642a484fd63dcd657cdd9c2bc13e8', './main.c' => '912c52d714d2edf8aa0d43601390ae7b', '__PBS_LIB_PATH/Rules/C.pm' => '9a3e1eeef74aa887e723a1e6135ba876', '__PBS_LIB_PATH/Configs/Compilers/compiler.pm' => 'ab5c5fba23b3b45f30bae03ac206e72f', '__PBS_LIB_PATH/Rules/C_depender.pm' => 'deadefe169fa3ecb947d6211d796c5a2', '__PBS_LIB_PATH/Configs/ConfigureProject.pm' => '6f692d3717eccef03452b3ac1882e2ce', '__PBSFILE' => '425fbf4ed9c8e7f3034b6e0acde50e71', '__PBS_LIB_PATH/MetaRules/FirstAndOnlyOneOnDisk.pm' => 'edc87f9805eec8a7dd61f33a70e770d0', '__PBS_LIB_PATH/Rules/BuildSystem.pm' => '353e53811d6c757bae8f654b95fbae11' };
AddFileDependencies() : PBS will compute an md5 for each file in the list you pass as argument and add it to the digest.
AddEnvironementDependencies(): PBS will add each environment variable you name in the list passed as argument. If the environment variable is not set, PBS will add the variable to the digest and give it the empty string value.
AddSwitchDependencies() : : PBS will add the variables and their values to the digest. Only Defined (-D) and User Defines (-u) can be added.
AddSwitchDependencies('-D*') ; # depend on all command line defines AddSwitchDependencies('-u*') ; # depend on all user variables from the command line AddSwitchDependencies('-u something', '-D debug', -D clean) ; # add only the given variables to the digest
AddVariableDependency() : This allows you to insert a variable name and it's value into the digest. For example, this could be used if you are cross compiling for an embedded platform from different OSes. The cross compilers would have different md5 on the OSes, so you can't add the cross compiler through AddFileDependencies().
my $compiler_version = GetCompilerNameAndVersion(...) ; AddVariableDependency('compiler_version' => $compiler_version) ;
AddNodeFileDependencies(qr/^.\/z0$/, 'pbs.html') ; AddNodeFileDependencies(qr/c/, 'pbs.pod') ; AddNodeVariableDependencies(qr/c/, 'a' => 1, 'b' => '2') ;
PBS can log into a remote computer through SSH or Telnet and execute a specific command.
use PBS::Shell_SSH ; PbsUse('Builders/HostedBuilder') ; my $shell = new PBS::Shell_SSH ( HOST_NAME => 'localhost' , USER_NAME => 'nadim' #~ , PROTOCOL => 1 # default is SSH2 #~ , REUSE_CONNECTION => 1 ) ; AddConfig C_COMPILER_HOST => $shell ; PbsUse('Rules/C') ; PbsUse('Configs/gcc') ; AddRule [VIRTUAL], 'all', ['*/all' => qw(source.o source2.o source3.o)], BuildOk() ;
In Rules/C:
$c_compiler_host = GetConfig('C_COMPILER_HOST') ; AddRuleTo 'BuiltIn', 'c_objects', [ '*/*.o' => '*.c' ] , HostedBuilder ( 'c_objects' , $c_compiler_host , "%CC %CFLAGS %CDEFINES %CFLAGS_INCLUDE -I%PBS_REPOSITORIES -o %FILE_TO_BUILD -c %DEPENDENCY_LIST" ) ;
PBS didn't work nice with IThreads.
PBS loads many modules
The monolitic data structure of PBS takes times to copy
The net result was a systemt was much (order of magnitudes) faster with a single thread.
Low level threads with synchronisation responsibility placed on the application would be PBS favorit solution. In the mean time an experimental implementation using fork has been started.
Trigger is a sytem to allow the current build to hook into other dependency trees. This can be used to (for example) synchronize libraries. the details are located in the library Pbsfile.
file://home/nadim/Dev/PerlModules/PerlBuildSystem-0.25/Pbsfiles/trigger/Pbsfile.pl
file://home/nadim/Dev/PerlModules/PerlBuildSystem-0.25/Pbsfiles/trigger/trigger.pl
Simultaneous variant build
Post build Commands
AddPostBuildCommand 'post build', ['all', 'a', 'b'], \&PostBuildCommandTest, 'hi' ; sub PostBuildCommandTest { my ($config, $name, $dependencies, $triggered_dependencies, $argument, $node) = @_ ; ... return(1, "PostBuildCommandTest OK.") ; }
Using PBS from PBS
file://home/nadim/Dev/PerlModules/PerlBuildSystem-0.25/Pbsfiles/frontend
[nadim@khemir PerlBuildSystem-0.25]$ tree PBSLib/ PBSLib/ |-- Builders |-- Configs |-- Dependers |-- MetaRules |-- Rules |-- UserBuild `-- Wizards |-- Breakpoint.pl |-- BuilderSub.pl |-- SimpleDependerSub.pl |-- StandardPbsfile.pl |-- Subpbs.pl |-- menu.pl `-- template.pl
file://devel/nailara/Pbs/Wizards/nailara_project.pl
file:///devel/nailara/Pbs/Wizards/nailara.pl
pbs -w nailara
On-line help
[nadim@khemir PerlBuildSystem-0.25]$ pbs -p Pbsfiles/test1/Pbsfile.pl -hu
HTML documentation for distribution
file://devel/nailara/Pbs/Rules/Nailara.pm
file://devel/nailara/Pbs/Rules/Nailara.pm.html
PrintError
PrintWarning or PrintWarning2
PrintInfo or PrintInfo2
PrintUser
PrintShell
PrintDebug
Colors for these functions can be defined through the command line or PBS_FLAGS. The "depend" step is indented. Output from user is also indented.
[nadim@khemir PerlBuildSystem-0.25]$ pbs -hs v v|verbosity=s: Used in user defined modules.
-- verbose is not used by PBS. It is intended for user defined modules.
I recommend to use the following settings:
Multiple verbosity switches can be given, they are store in {__PBS_CONFIG}{ VERBOSITY}.
C-Tech build system uses colorgcc. http://www.mindspring.com/~jamoyers/software/colorgcc/
PBS is a three pass sytem.
You can take over after any pass you'd like to
With some work you can even fiddle with the build sequence
Using default Build():
pbs -p Pbsfiles/user_build/user_build.pl -no_user_build -tt -tno -fb -no_digest x.lib
Using your own Build():
pbs -p Pbsfiles/user_build/user_build.pl -tt -tno -no_digest -dbsno x.lib
Rules ... Rules ... #------------------------------------------------------------------------------- PbsUse('UserBuild/BuildSequenceMiner') ; PbsUse('UserBuild/Multiple_O_Compile') ; sub Build { my ($build_result, $build_message) = BuildSequenceMiner ( [@_] , [\&Multiple_O_Compile] ) ; PrintInfo("Build done.\n") ; return($build_result, $build_message) ; }
[nadim@khemir argus2_test]$ time pbs
uses PRF
verbose
speed
Pbs offers switches to lower output verbosity.
No header: -nh
No step info: -nsi
Silent command: -sc
Silent command output: -sco
No build header: -nbh
Silent depend info: -sdi
One PBS requirements was to help the PBsfile writer to find errors in the build system he defines. PBS has an army of switches to help you get insight in how the dependency tree is constructed.
-display_pbsuse
-display_used_rules_name_only
-display_config
-display_dependencies, -a and -display_dependencies_regex
-text_tree and tree_node_triggered_reason
-display_build_sequence
-display_search_info
-display_rules -display_used_rules and -display_rule_definition
-display_dependency_regex
-display_digest
-f
-ni, -bi and -bni
-o
-create_log
-dump
gtg
gtg_p
file://home/nadim/Dev/PerlModules/PerlBuildSystem-0.25/Pbsfiles/config/tree_package.png
gtg_cn
file://devel/nailara/projects/argus2_test/tree_cn.png
gth_html and gtg_html_frame
file://devel/nailara/projects/argus2_test/html/index.html
gtg_tn
gtg_config and gtg_config_edge
file://home/nadim/Dev/PerlModules/PerlBuildSystem-0.25/Pbsfiles/config/tree_config.png
gtg_pbs_config and gtg_pbs_config_edge
file://home/nadim/Dev/PerlModules/PerlBuildSystem-0.25/snapshots/00000.png
#DEBUG HOOK (see PBS::Debug) my %debug_data = ( TYPE => 'BUILD' , CONFIG => $file_tree->{__CONFIG} , NODE_NAME => $file_tree->{__NAME} , NODE_BUILD_NAME => $build_name , DEPENDENCIES => \@dependencies , TRIGGERED_DEPENDENCIES => \@triggered_dependencies , ARGUMENTS => \$arguments , NODE => $file_tree ) ; #DEBUG HOOK, jump into perl debugger is so asked $DB::single = 1 if(PBS::Debug::CheckBreakpoint(%debug_data, PRE => 1)) ;
AddBreakpoint ( 'hi' , DEPEND => 1 , PRE => 1 #~ , USE_DEBUGGER => 1 #~ , ACTIVE => 1 , ACTIONS => [ sub { PrintDebug "Hi there.\n" ; } ] ) ;
types:
DEPEND
INSERT
TREE
BUILD
POST_BUILD
Sub types
PRE
POST
TRIGGERED
filters :
RULE_REGEX
NODE_REGEX
PACKAGE_REGEX
PBSFILE_REGEX
PBS takes 6 seconds, in a test project, to do nothing vs 0.2 for gmake. OK, gmake doesn't do what PBS does but for the day to day work, 6 seconds are just too much!
We already try to not write unoptimal code
*Optimizing research code is a non sense
Optimizing is long and dificult
A day of optimizing work saved 1 second! (a single line of code)
[nadim@khemir argus2_test]$ rm nadim_out/nisse/nisse.o [nadim@khemir argus2_test]$ time pbs -warp
http://www.grc.nasa.gov/WWW/PAO/warp.html
What can be done to run faster?
Caching perl code generated by Pbsfiles
Partial sub tree regeneration
Parallel depend step
Build in the background while editing
<Your crazy ideas here>
size
number of modules
used modules
Cwd.pm 2.06 Devel/Cycle.pm 1.01 Devel/Size.pm 0.58 Digest/MD5.pm 2.27 File/Basename.pm 2.71 File/Copy.pm 2.05 File/MkTemp.pm 1.0.6 File/Path.pm 1.05 Getopt/Long.pm 2.32 List/Util.pm 1.07 Pod/Parser.pm 1.13 Scalar/Util.pm 1.07 Term/ANSIColor.pm 1.05 Term/Size.pm 0.2 Text/Balanced.pm 1.95 Text/Tabs.pm 98.112801 Text/Wrap.pm 2001.0929 Tie/Hash.pm 1.00 Tie/IxHash.pm 1.21 Time/HiRes.pm 1.2
License
OO/ re-entrency
Documentation / tests
Energy/ideas
Find people interrested in PBS
Release
The fun stuff
Parallel build
Distributed build in heterogene environment
Clustering
Configuration management
Centralized build point (web)
Warp 2
Distributed Pbsfiles
Tackle bigger build projects
Optimize code
More Perlishness
<Your ideas here>
Contact: nadim@khemir.net.
112 POD Errors
The following errors were encountered while parsing the POD:
Unknown directive: =slide
Can't have a 0 in =over 0
You forgot a '=back' before '=head1'
Expected '=item *'
Non-ASCII character seen before =encoding in 'Ladulås--------------------------------------------------'. Assuming CP1252
To install PBS::Log, copy and paste the appropriate command in to your terminal.
cpanm
cpanm PBS::Log
CPAN shell
perl -MCPAN -e shell install PBS::Log
For more information on module installation, please visit the detailed CPAN module installation guide.