Monzool’s Personal Publishing

AFTER SOME TIME of using tarballing as a source revision control system, I thought it was about time getting back to using a real tool for managing my source code.

I’ve then used Perforce as my VCS for some time now. At work we are using ClearCase and from what I could gather form the Perforce documentation it seemed that Perforce would operate very much like ClearCase. However its clear now, that I’m not Perforce “compatible”. I find Perforce very unintuitive, and after a second time of having to resort to clearing my repository, after another fine [workspace] mess I’ve gotten my self into, Its clear that its time to try another system.

My new requirements to a VCS is that it must be FOSS and run on Linux, MAC OS X, OpenSolaris and Windows (I don’t use Windows but I can’t exclude that the need won’t come up some day). I also want to be able to work on the source without being connected to the repository server.

As this comparison chart on Wikipedia shows; one thing is sure: The world is not in lack of source revisioning systems.

After a lot of googling it seems that the hot contenders of the time are: Bazaar, Git, darcs and Mercurial.

Choosing the one over the other is extremely difficult as not much differentiate them from each other. They all have their niches where they excel compared to the others, but from what I can understand from their feature list, they will all be more than enough for my VCS needs.
Two candidates I could eliminate quite easy. The FOSS requirement is because I want to have the possibility to look around, and perhaps poke a bit, in the code. Git is primarily written in C, but my patience just don’t stretch enough to look into another C project written by a bunch of kernel hackers. Darcs is written in Haskell, and regardless of the “hotness” of Haskell I don’t have any intentions of learning that language. Thus candidates remaining are Bazaar and Mercurial.

Bazaar and Mercurial are so alike that its hard to believe that one did not just fork from the other. Searching the internet gave no reason to believe that the one is inferior to the other. So which is better? I think that is impossible to determine without hands on experience from both systems – and from what I can read on the internet, even this is not always enough to prefer the one over the other. Actually this makes me question the rationale of having two projects so similar. Competition is good, but if the one does not stand out from the other, why not join forces?

Lots of people talk of Bazaar and/or Marcurial. Here’s a page that’s pro Bazaar: My merry five minutes with Bazaar. And here’s a page that’s pro Mercurial: Distributed SCMs: Be Smart, Use the Bleeding Edge

To summarize my internet readings (here is a dedicated on-topic site), then I didn’t find any clearcut arguments for Mercurial over Bazaar other than perhaps speed and maturity level. Bazaar on the other hand seems to be regarded as having slightly more user friendly use-cases.

Mercurial have support from prominent projects like OpenSolaris, Mozilla, Plan 9 and Xen. Bazaar have Ubuntu. Not sure the merits actually mean anything though.

This site gave a good comparison of some high profiled VCS. Of my two candidates it only included Mercurial, but the comparison did include ClearCase which made it clear that Mercurial offered more or less the same feature set as provided by ClearCase.

The options seems clear. Keep tarballing, roll the dices or give both Bazaar and Mercurial a try… hmm I better give them a try ;-)

SOURCE CODE COLORING in WordPress is thankfully a pretty simple task, thanks to the WordPress plugin system. This allows for installing filters to do the coloring and formatting automatic. Several plugins exists for WordPress to do source code coloring, but (for now?) I’ve settled on the Wp-Syntax plugin. Not that its necessary better that the other options, but it does a pretty fine job, and I like the layout it presents.

However the default coloring scheme in Wp-Syntax is not to my taste, so I had to do some tweaking. Wp-Syntax uses the popular GeShi Highlighter as backend, and GeShi is highly customizable.

Wp-Syntax splits the source area on two columns with one optional column for line numbering and one for the source code. The formatting of the line numbering column and the boxes surrounding both columns are both configured separate from GeShi. The only thing required to style the non-GeShi things is to apply the desired formatting in the main css file (note that the Wp-Syntax css may override the GeShi css, e.g. color). All classes are documented on the Wp-Syntax notes page. Below is a snippet from this site configuration.

.wp_syntax {
  color: #404040;
  background-color: #F9F9F9;
  border: 1px solid silver;
  overflow: auto;
  margin-top: 10px;
  margin-bottom: 10px;
}
 
.wp_syntax .line_numbers {
  text-align: right;
  background-color: #46555D;
  color: orange;
  overflow: visible;
}

The GeShi part that formats the actual source code, requires a few more simple steps. The Wp-Syntax notes page, explains how to modify changes to the default GeShi color settings. I don’t find the method descriped that optimal. I would rather configure the coloring in sites css file. The steps required for this is described in the GeShi documentation. First step is to enable the use of css classes. The documentation states that this should be enabled right after instantiation of the GeShi object. This can be done by adding lines 2 and 3 in the Wp-Syntax file wp-syntax.php.

1
2
3
4

    $geshi = new GeSHi($code, $language);
    $geshi->enable_classes();
    $geshi->get_stylesheet();
    $geshi->enable_keyword_links(false);

Remaining is the actual css contents for defining the colors. Here the team behind GeShi excell by supplying a php script for generating the css code. If downloading the GeShi source the script is available as /geshi/contrib/cssgen.php. Following the three easy steps in the script generates the css ready to paste in the main css file.

Thats it! Colors tweaked.

AT MY DAILY work we use MIB’s extensively for product configurations. This often brings up the need to recreate a certain configuration sequence. If the scequence is not included in the normal test suite, and the sequence is more that just a couple of OID’s, I usually create a small Lua script to perform the MIB configuration.

Currently I am using Lua for scripting and the Net-Snmp command line tools as backend.

Here follows a simplified example of my normal Lua scripts. First step is to define the OID’s to use. I collect these in a file called Mibs.lua

1
2
3
4
5
6
7
8
9
10
11

#!lua
--[[
      Net-Snmp datatypes:
         i: INTEGER, u: unsigned INTEGER, t: TIMETICKS, a: IPADDRESS
         o: OBJID, s: STRING, x: HEX STRING, d: DECIMAL STRING, b: BITS
         U: unsigned int64, I: signed int64, F: float, D: double
--]]
 
-- Standard MIB
Mib_sysName     = { "1.3.6.1.2.1.1.5", "s" }
Mib_sysLocation = { "1.3.6.1.2.1.1.6", "s" }

This file defines to tables containing each a OID prefix and the datatype.

Invoking the SNMP calls to perform the MIB manipulations is handled by a Lua class that wraps the Net-Snmp command line tools. This class is located in a file Snmp.lua

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74

#!lua
--[[
      SNMP by using Net-SNMP executeables
--]]
 
-- Mibs.lua provides MIB definitions.
mibs_lua = loadfile("Mibs.lua") -- Include the file in compilation
mibs_lua()                      -- Define its contents
 
 
CSnmp = {}
 
function CSnmp:new(o)
  -- Create object
  o = o or {}
 
  -- Set 'self' as prototype for the table (object)
  setmetatable(o, self)
  self.__index = self
 
  -- Initialize
  self:Initialize()
  return (o)
end
 
-- Initialize
function CSnmp:Initialize()
  self.dut  = "10.0.0.17"          -- IP of DUT
  --self.attr = "-r 3 -t 500 "     -- Three retry's with timeout of 500[ms]
  self.attr = "-r 0 "              -- No retry
  self.opt  = "-c private -v 2c "
  self.set  = "snmpset.exe "       -- Net-Snmp commandline tool for 'SET'
  self.get  = "snmpget.exe "       -- Net-Snmp commandline tool for 'GET'
end
 
--[[ MIB set.
     Set an OID for an optional line with a value.
     IN:    oid:       OID to set. Format is { "oid prefix", "type" }.
            component: Component specifier.
            value:     Value to set.
--]]
function CSnmp:Set(oid, component, value)
 
  -- Comprise OID prefix and component specifier (if any)
  local oidFull = oid[1] .. component;
 
  -- Append identify value type
  local cmdType = " " .. oid[2] .. " "
 
  -- Wrap string types in "'s
  local cmdValue
  if cmdType == " x " or cmdType == " s " then
    cmdValue = "\"" .. value .. "\""
  else
    cmdValue = value
  end
 
  -- Collect command line
  local cmd = self.set .. self.attr .. self.opt .. self.dut .. " " ..
          '"' .. oidFull .. '"' .. cmdType .. tostring(cmdValue)
 
  -- Run the external program with parameters
  --print(cmd)  -- Debug
  os.execute(cmd)
end
 
--[[ MIB get.
     Get an OID value.
     IN:    oid:    OID to set (including line if required).
--]]
function CSnmp:Get(oid)
  local cmd = self.get .. self.opt .. self.dut .. " " .. oid
  os.execute(cmd)
end

The table CSnmp defines a function new that provides object allocation and configuration for the Net-Snmp command line tools. Two methods are defined in the class: a Set and a Get. The method Set accepts a table of the kind defined in the file Mibs.lua. The component parameter is for adding “.0″ or perhaps a table index to the OID. The simpler Get member accepts a full specified OID only.

The files created so far can be reused by all configuration scripts. An example of a configuration script could be to request the (configured) location of the device, and the set a new location.

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16

#!lua
 
-- Snmp.lua provides SNMP GET/SET functionality.
snmp_lua = loadfile("Snmp.lua") -- Include the file in compilation
snmp_lua()                      -- Define its contents
 
snmp = CSnmp:new()
 
print("GET Location:")
snmp:Get(Mib_sysLocation[1] .. ".0")
 
print("SET Location:")
snmp:Set(Mib_sysLocation, ".0", "Mars")
 
print("GET Location:")
snmp:Get(Mib_sysLocation[1] .. ".0")

Running the script produces the following output:

>sys.lua
  GET Location:
  iso.3.6.1.2.1.1.6.0 = STRING: "Moon"
  SET Location:
  1.3.6.1.2.1.1.6.0: "Mars"
  GET Location:
  iso.3.6.1.2.1.1.6.0 = STRING: "Mars"

The scripts started out as Python scripts. I then ported them all to Ruby just to compare the two languages. Finally I ported the scripts to Lua. For basic scripting, there is actually not much difference between Python, Ruby or Lua, other than perhaps the storage needs. The Lua interpreter and library footprint is about 200KB which is only a fraction of the space occupied by the ActivePython or ActiveRuby allowed for installation on our company computers. The small size is handy if moving the scripts to run directly on the closed network test equipment.

REPLACEMENT NEW CAME to mind, when working on a C++ project of mine. In this project some large objects of short lifetime led to a concerns about memory fragmentation on the targeted embedded platform that have only sparse memory ressources. The life cycle of the objects was to create an object, then fill the large data member structures, store data for a while until time for transmitting the data and terminate the object. The object data where always different so they could not be reused, and the lifetime of each object was inditerminable. However it was certain that only a known maximum amount of objects would exist at the same time.

This senario made me think of a rarely used memory allocation functionality called placement new that would handle the above concerns perfectly. With this variant of new you allocate a memory pool of fixed memory address and size, from which allocations can be made from. This way placement new allows reuse of the same memory again and again and thus containing any memory fragmentation within the disposable memory pool.

Below is a simple example of how to use the placement new functionality.

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57

#include <iostream>
#include <iomanip>
#include <new>
#include <stdio.h>
using namespace std;
 
class C
{
  public:
            C() : data(0x11) { ; }
            ~C() { data = 0x22; }
 
            char data;
};
 
int main(int argc, char *argv[])
{
  const int PoolItems = 6;
  const int PoolSize  = PoolItems * sizeof(C);
 
  // Allocate a memory pool
  char *pool = new char[PoolSize];
  memset(pool, 0x00, PoolSize);
 
  // Create some pointers to access pooled objects
  C *pC[PoolItems] = { 0 };
 
  // Allotate objects at "arbitary" addresses in the pool
  pC[1] = new (pool + sizeof(C)*1) C;
  pC[4] = new (pool + sizeof(C)*4) C;
 
  // Display constructed data
  for (int i = 0; i < PoolSize; i++)
    cout << setfill('0') << setw(2) << std::hex << (int)pool[i] << " ";
  cout << endl;
 
  // Explicit destruction of objects in pool
  for (int i = 0; i < PoolItems; i++)
  {
    if (pC[i] != NULL)
    {
      pC[i]->~C();  // Explicit destructor call
      pC[i] = NULL;
    }
  }
 
  // Display destructed data
  for (int i = 0; i < PoolSize; i++)
    cout << setfill('0') << setw(2) << std::hex << (int)pool[i] << " ";
  cout << endl;
 
  // Release memory in pool
  delete [] pool;
  pool = NULL;
 
  return EXIT_SUCCESS;
}

Explanation: First a memory pool is created that fits a fixed amount of the designated class objects. Two object are then created within pool slots 1 and 4. The simple byte sized class C has a contructor initialization of the member data.

The first printing of the pool will display the following contents:

00 11 00 00 11 00

This indicates that two slots are allocated with objects of the class C.

Destructing the objects are done by calling the destructor explicitly, NOT by normal delete action (as the memory pool original was created as a char array, the memory it occupies is tagged as a char array that must be deleted the approriate array delete operation). After destruction, the pool contents is displayed again, and it is seen that the same memory has been rewritten by the destructor assigning:

00 22 00 00 22 00

Finally the memory pool is deleted and the memory is released.