Category Archives: Smalltalk

Smalltalk stuff comes here

Oz object spaces in Esug 2013

Hi!

This little post is to add a summary of what I’ll be showing in the upcoming ESUG conference. I’ll present a paper about my work in the workshop and present the recovery tools of Oz in the innovation awards :).

For the paper

I’ll present the ideas and implementation of Oz object spaces. Especially the metacircular problems it solves, and a bit on the vision we (I and my supervisors have on this line of work). If you want to read the paper, you can go to:

http://rmod.lille.inria.fr/archives/papers/Poli13a-IWST13-ObjectSpacesVirtualization.pdf.

For the lazy, I paste here the abstract:

Reflective architectures are a powerful solution for code browsing, debugging or in-language process handling. However, these reflective architectures show some limitations in edge cases of self-modification and self-monitoring. Modifying the modifier process or monitoring the monitor process in a reflective system alters the system itself, leading to the impossibility to perform some of those tasks properly. In this paper we analyze the problems of reflective architectures in the context of image based object-oriented languages and solve them by providing a first-class representation of an image: a virtualized image.
We present Oz, our virtual image solution. In Oz, a virtual image is represented by an object space. Through an object space, an image can manipulate the internal structure and control the execution of other images. An Oz object space allows one to introspect and modify execution information such as processes, contexts, existing classes and objects. We show how Oz solves the edge cases of reflective architectures by adding a third participant, and thus, removing the selfmodification and self-observation constraints.

For the innovation awards

I’ve prepared a demo and with that I recorded a little video showing it. You can see it in:

Also, we have a sexy logo!

Oz logo

Oz logo, created by my friend Ximena Fernandez
Copyright (C) 2013 Ximena Fernandez

I asked my friend Ximena to make a logo for my projects, even pushing her with the deadline of ESUG 2013 (sorry xime ;), and she created this one, that I really like.

So, thanks again Xime!

Now, waiting for next week @ ESUG, to taste the ESUG beers ;).
Enjoy!
Guille

Advertisements

Customizing ZeroConf scripts

As you may already know, ZeroConf scripts are bash scripts that ease the installation of a Pharo environment. A funny thing about these ZeroConf scripts is that they are seen as bash scripts by a bash terminal, and as simple and minimal html pages by a web browser. These scripts are extensively used to simplify the configuration of Pharo CI jobs. They allow you to easily download many versions of the Pharo image and VM.

As I’m working for my Phd, and have a custom version of my virtual machine and image, and also want to make use of the advantages our CI server provides, I wanted to build my own ZeroConf scripts specialized for my needs. I also heard recently on the pharo mailing list that there was some work on customizing ZeroConf scripts[1] for Moose[2]. So I wanted to do it as well for my project :).

[1] http://get.moosetechnology.org/
[2] http://www.moosetechnology.org/

Downloading the ZeroConf package

The ZeroConf scripts are generated automatically by the ZeroConf Pharo package. You can find this package in [2]. To download the current version, you just have to execute the following piece of code in a workspace:

Gofer it
	smalltalkhubUser: 'Pharo' project: 'ZeroConf';
	package: 'ZeroConf';
	loadVersion: '1.0'

This code snippet will install in your image the ZeroConf package for the 1.0 version, containing the script generator, and some tests that are not currently working :).
[2] http://www.smalltalkhub.com/#!/~Pharo/ZeroConf

Getting what’s inside ZeroConf

The ZeroConf package is pretty small and simple. There is an abstract class AbstractZeroConfBashScript implementing most of the script generation and bash writing utils. Its subclasses will implement the concrete script generation. Current implementation includes three main classes below the AbstractZeroConfBashScript hierarchy, implementing a composite pattern:

  • ZeroConfImageScript: Generates scripts in charge of downloading image files.
  • ZeroConfVMScript: Generates scripts in charge of downloading Virtual Machine files (and source files).
  • ZeroConfCombinedScript: Generates scripts that combine several scripts. It will point to its combined scripts and downloading it means to download them all.

zeroconf

Customizing our ZeroConf scripts

As you can see in the picture, in order to customize the ZeroConf scripts, you have to create your own subclasses and overriding the correct hooks.

Customizing an image script

A custom image script is defined by a subclass of ZeroConfImageScript. ZeroConfImageScript already defines some image common behavior, such as the release number, which we will use in our script.

ZeroConfImageScript subclass: #OzZeroConfImageScript
	instanceVariableNames: ''
	classVariableNames: ''
	poolDictionaries: ''
	category: 'OurZeroConf'

Then we override the methods that tell some information about the files to download:

  • OzZeroConfImageScript>>imageName: The name of the image and changes files
  • OzZeroConfImageScript>>imageUrl: The url of the image zip, instance of ZnUrl
  • OzZeroConfImageScript>>defaultBaseName: The baseName of the script
  • OzZeroConfImageScript class>>baseUrl: The base url where the scripts are found, instance of ZnUrl
  • OzZeroConfImageScript class>>fileBaseUrl: The base url where the files the scripts will download are found, instance of ZnUrl

I implemented them in my class as follows

OzZeroConfImageScript>>imageName
	^'Oz'

OzZeroConfImageScript>>imageUrl
	^ self fileBaseUrl / 'image' / self release asString / self imageFileName / 'download'

OzZeroConfImageScript>>imageFileName
	^'Oz-image-', self release asString , '.zip'

OzZeroConfImageScript>>defaultBasename
	^ self imageName, self release

OzZeroConfImageScript class>>fileBaseUrl
	^ 'https://sourceforge.net/projects/ozobjectspaces/files' asZnUrl

OzZeroConfImageScript class>>baseUrl
	^ self fileBaseUrl / 'get'

I also extended my script so it generates a custom html title and uses my combining script when combining:

OzZeroConfImageScript>>htmlTitle
	^ self imageName, ' Zeroconf Script'

OzZeroConfImageScript>>defaultCombiningScript
	^ OzZeroConfCombinedScript

Finally, I created a convenience method for creating a script corresponding to the 1.0 version of my custom image.

OzZeroConfImageScript class>>oz10
	^self new
		release: '1.0';
		yourself

Now you can try generating your script in a workspace,

OzZeroConfImageScript oz10 generate

and see the generated results in your working directory!

Customizing a vm script

A custom vm script is defined by a subclass of ZeroConfVMScript. ZeroConfVMScript defines, as its image friend, some vm common behavior, such as the release number and virtual machine type(i.e., if it is a jitted vm or not), which we will use in our script.

ZeroConfVMScript subclass: #OzZeroConfVMScript
	instanceVariableNames: ''
	classVariableNames: ''
	poolDictionaries: ''
	category: 'OurZeroConf'

Then we override the methods that tell some information about the files to download:

  • OzZeroConfVMScript>>binaryName: The name of the vm binary name
  • OzZeroConfVMScript>>binaryNameLinux: The name of the vm binary name in linux, which tends to be different
  • OzZeroConfVMScript>>vmUrl: The url of the vm zip, instance of ZnUrl
  • OzZeroConfVMScript>>defaultBaseName: The baseName of the script
  • OzZeroConfVMScript class>>baseUrl: The base url where the scripts are found, instance of ZnUrl
  • OzZeroConfVMScript class>>fileBaseUrl: The base url where the files the scripts will download are found, instance of ZnUrl

I implemented them in my class as follows

OzZeroConfVMScript>>binaryName
	^'ozstack'

OzZeroConfVMScript>>binaryNameLinux
	^self binaryName

OzZeroConfVMScript>>vmUrl
	^self fileBaseUrl asString, '/vm/', self release asString,'/', self vmFileName, '/download'

OzZeroConfVMScript>>vmFileName
	^'OzVm-${OS}-', self release asString , '.zip'

OzZeroConfVMScript class>>fileBaseUrl
	^ 'https://sourceforge.net/projects/ozobjectspaces/files' asZnUrl

OzZeroConfVMScript class>>baseUrl
	^ self fileBaseUrl / 'get'

I also extended my script so it uses my combining script when combining:

OzZeroConfVMScript>>defaultCombiningScript
	^ OzZeroConfCombinedScript

Finally, I created a convenience method for creating a script corresponding to the 1.0 version of my custom vm.

OzZeroConfVMScript class>>ozvm10
	^self new
		type: 'oz';
		release: '1.0';
		yourself

Now you can try generating your script in a workspace,

OzZeroConfVMScript ozvm10 generate

and see the generated results in your working directory!

Customizing a combined script

A combined script is the one we use to combine several scripts. It is defined by a subclass of ZeroConfCombinedScript.

ZeroConfCombinedScript subclass: #OzZeroConfCombinedScript
	instanceVariableNames: ''
	classVariableNames: ''
	poolDictionaries: ''
	category: 'OurZeroConf'

Then we override the methods that tell some information about the files to download:

  • OzZeroConfCombinedScript class>>baseUrl: The base url where the scripts are found, instance of ZnUrl
  • OzZeroConfCombinedScript class>>fileBaseUrl: The base url where the files the scripts will download are found, instance of ZnUrl

I implemented them in my class as follows

OzZeroConfCombinedScript class>>fileBaseUrl
	^ 'https://sourceforge.net/projects/ozobjectspaces/files' asZnUrl

OzZeroConfCombinedScript class>>baseUrl
	^ self fileBaseUrl / 'get'

As you can see, my methods baseUrl and fileBaseUrl are always the same in all scripts. I extracted them into another class later, but keep the code here as is for clarity.

I also extended my script so it uses my combining script when combining and the html title:

OzZeroConfCombinedScript>>defaultCombiningScript
	^ OzZeroConfCombinedScript

OzZeroConfCombinedScript>>htmlTitle
	^ self scripts first htmlTitle

Integrating everything and automating generation

As I already showed you, every script understands the message #generate to generate itself. However, we may want to generate many scripts, and combine them. The ZeroConf infrastructure already provides for that the ZeroConfCommandLineHandler. The ZeroConfCommandLineHandler is a command line handler that knows which are the scripts we want to generate, combines them appropriately and generates them. So we will subclass from ZeroConfCommandLineHandler and specialize it to fulfill our needs.

ZeroConfCommandLineHandler subclass: #OzZeroConfCommandLineHandler
	instanceVariableNames: ''
	classVariableNames: ''
	poolDictionaries: ''
	category: 'OzZeroConf'

Once we have it, we configure it as a command line handler specifying its command name and description:

OzZeroConfCommandLineHandler class>>commandName
	^ 'ozzeroconf'

OzZeroConfCommandLineHandler class>>description
	^ 'Generate Oz zeroconf bash scripts'

And finally we specialize it to tell it about our scripts:

OzZeroConfCommandLineHandler>>defaultScript
	^ self defaultImage, self defaultVM

OzZeroConfCommandLineHandler>>defaultImage
	^ OzZeroConfImageScript oz10

OzZeroConfCommandLineHandler>>defaultVM
	^ OzZeroConfVMScript ozvm10

OzZeroConfCommandLineHandler>>imageScripts
	^ { 
		OzZeroConfImageScript oz10.
	}

OzZeroConfCommandLineHandler>>vmScripts
	^ { 
		OzZeroConfVMScript ozvm10
	}

OzZeroConfCommandLineHandler>>indexScriptExamplesHtml
	^ ''

Now we have our command line handler, we can test it and make it generate our scripts:

OzZeroConfCommandLineHandler new
	commandLine: CommandLineArguments new;
	generateScripts

Finally, if you have all this code in your image, you can just activate it through the command line thanks to the command line handler!

./pharo PharoMyZeroConf.image ozzeroconf

And look at the results. Upload your files, archive them, and use them :).

Guille

Installing Pharo in many flavors

Buon giorno!

In the recent times, there appeared many many ways to leverage the installation and deploy of Pharo applications. These installation approaches enhance significantly the experience of using Pharo, by simplifying either dependency management with OS libraries, enabling to write deploy bash scripts or loading prebuilt images for any (and many) taste(s).

However, if you are not in the Pharo mailing lists, you probably have not heard about many of these installation mechanisms, and therefore, you cannot enjoy them. So, let’s summarize a bit some of these mechanisms, at least the ones I know. If you know some more, contact me so we can include it.

Manual download from the webpage

Downloading Pharo manually is the easiest but more primitive approach. Proceed to the download page [1] and download the flavor of Pharo you like the most. You will find in here the 1.3, 1.4 and 2.0 releases, plus the option to load the latest (still in development) version of Pharo 3.0.

Focusing on what is available for Pharo 2.0, you can either install

  • under the category “Pharo Installers”: a package specific for your operative system containing both virtual machine and the image with the runtime and development environment
  • under the category “Custom Downloads”: the possibility to download them by separate. This option is useful if you already have a virtual machine and only want a new image to play with.

[1] http://www.pharo-project.org/pharo-download

Manual download from the file server

In the Pharo file server[2] you will find available the virtual machine and image releases as well as other resources to download. You can use these urls to create your custom download scripts.

[2] http://files.pharo.org/

Virtual Machine PPA for Ubuntu linux

There is a PPA available for Ubuntu users (probably it works also for any distribution using apt-get package manager) which is in charge of downloading the virtual machine and its dependencies, simplifying its installation and deploy. We thank Damien Cassou for taking finally the initiative of creating the PPA!

#install the PPA repository
sudo add-apt-repository ppa:pharo/stable
sudo apt-get update

#install pharo vm core
sudo apt-get install pharo-vm-core

#install pharo vm for desktop (with graphical dependencies)
sudo apt-get install pharo-vm-desktop

ZeroConf scripts

The ZeroConf scripts[3] are already built bash scripts easing the download and installation of pharo. They are scripts served by get.pharo.org which can be parameterized for getting the pair vm/image you want.

Their usage, as written in the ZeroConf webpage can be resumed as

curl url | bash
#or if curl is not available:
wget -O- url | bash

where url is replaced by the formula vmVersion|imageVersion|vmVersion+imageVersion

For example, some valid usages of ZeroConf are

#downloading latest 3.0
curl get.pharo.org/alpha | bash

#downloading stable 2.0 + vm
curl get.pharo.org/20+vm | bash

#downloading latest non stable vm
curl get.pharo.org/vmLatest | bash

You can look for the valid values in the ZeroConf page [3]. These scripts are currently heavily used by the ci infrastructure of pharo. We thank Camillo Bruni for pushing this harder!

In fact, this is the way I download my own images right now, because the url is easy to memorize and using the terminal is pretty straightforward.

[3] http://get.pharo.org/

Pharo Launcher

The Pharo Launcher is an application to download and manage prebuilt and custom Pharo images. Below I paste the release notes from the first release:

“Erwan and I are proud to announce the first release of the Pharo
Launcher, a cross-platform application that

– lets you manage your Pharo images (launch, rename, copy and delete);
– lets you download image templates (i.e., zip archives) from many
different sources (Jenkins, files.pharo.org, and your local cache);
– lets you create new images from any template.

The idea behind the Pharo Launcher is that you should be able to
access it very rapidly from your OS application launcher. As a result,
launching any image is never more than 3 clicks away.

Download: https://ci.inria.fr/pharo-contribution/job/PharoLauncher/PHARO=30,VERSION=bleedingEdge,VM=vm/lastSuccessfulBuild/artifact/PharoLauncher.zip

Please report bugs on the ‘Launcher’ project at https://pharo.fogbugz.org

You can contribute to this project. All classes and most methods are
commented. There are unit tests. Please contribute!

Source code: http://www.smalltalkhub.com/#!/~Pharo/PharoLauncher
CI: https://ci.inria.fr/pharo-contribution/job/PharoLauncher

Pharo Launcher screenshot

Pharo Launcher screenshot

The Pharo launcher is an initiative of Erwan Douaille and Damien Cassou. And of course, you can contribute to it. In their release notes they added some points they would like to enhance in this project:

  • check if a template is already downloaded before downloading it
  • add a preference mechanism (for, e.g., quit after launch, definition of your own template groups, location of downloaded templates and images)
  • put the launcher in the Pharo Ubuntu package so that the launcher becomes a registered application of the system (https://launchpad.net/~pharo/+archive/stable)
  • make sure the pharo launcher does not load your personal scripts (like fonts and MC configuration)
  • add a toolbar to enhance the discoverability of the features (currently everything is in contextual menus)
  • make sure rename and copy actions propose default values
  • make sure no debugger pops up when a user press cancels or enter an invalid name
  • propose multiple kinds of sorting (last used, most frequently used, alphabetically on the name)
  • give some information about each template (build date, pharo version)

Conclusion

Pharo is growing, and getting sexy. And now you have easy deploy, and it will get only easier in the future. What are you waiting?

#Just do this!
curl get.pharo.org/20+vm | bash
./pharo-ui Pharo.image

Bootstrap revival – the basics

In the last time I was working on the system bootstrap again, trying to enhance the process, lower the imposed limitations… And I have a pretty new version. I’ll present here a summary of what I’ve learnt in the last time:

  • A review on the process steps, with exemplar code snippets
  • Some of the new key ideas and new infrastructure
  • And improvements over the last version :)

What are we going to bootstrap

For the sake of simplicity, we will not bootstrap a full Pharo (because that would include preparing for example Morphic to be bootstrapped, or to be loaded/unloaded). Instead, we will bootstrap an adapted version of MicroSqueak from John Maloney, which we re-baptized as Pharo Candle. Pharo Candle has only 50 classes, as you can see in the Github repo, and limited features. All classes from Pharo Candle has a PC prefix, which is not important for the bootstrap, since it will not let be name collisions.

To understand the rest of the post, the following is important: we will take a textual and static definition of the system (methods, classes…), and load it into our image. For that we will parse the files and include them as definition objects. These definition objects will ease the access to the data needed instead of accessing lots of lookup and symbol tables and stuff in a procedural way…

Some infrastructure basics

The last bootstrap implementation was very coupled to Pharo and its internal implementation. To create objects, to initialize classes, we relied on the existing system, on sending messages, on the VirtualMachine interpreter and infrastructure. So the first step for this new version was to decouple that. Decoupling the existing system from the new system that is about to be created. To do that we encapsulate the state of the new system inside an object, which we call an object space.

An object space reifies the new system. It is an object that understand messages such as “create an object”, “translate this string to a string of the new system”, “register this object as a class in yourself” or even “execute this piece of code”. This object space lets us structure the creation of the new system in a more comfortable way.

Our next concern, is not installing objects from the existing system into the new system. We want the new system to be transitively closed. So, the best is not to have direct references from the existing system to the new one. The best is to control those references carefully. Then, every time an object space gives you a reference to one of his objects, he really gives you a proxy. That proxy allows you to manipulate, at some level, the object inside the object space. A proxy can give and revoke permissions on the inner object, and make transformations or validations when necessary to keep the model consistent. Since these proxys may perform meta-operations on the new system objects, we end up calling them mirrors at some point.

Finally there are VirtualMachine limitations when executing code on the new space. We introduced a new piece in the puzzle to overcome them: our own language-level interpreter. In this particular case, we are using an AST interpreter, but we could, if available, use any other kind of interpreter. The important thing about using our own interpreter is controlling the semantics of the new system, and leverage the VM’s limitations.

Where do we start?

When the bootstrap starts, there is nothing. There are no objects, no structures, nothing. So we have to build everything from the start. And the question is… where do we start?

Our system is composed by objects, and thus, we need to create objects. And every new object may have pointers, which are initially pointing to nil. But what if there is no nil in our system? So, let’s build a first nil, so all objects created later can point to this nil object.

Creating the first object

As we decided before, we create our first nil object. However, there is a question that arises when we try to create nil. How can we create an object without a class? The answer is, so far, that until we have classes we will create objects without classes. We will not care about their classes and we will solve that later, once classes are created. Fortunately, this problem is present with very few objects.

theNil := objectSpace 
    createObjectWithFormat: undefinedObjectDefinition nilFormat.

Since we have no class to create nil, we have to specify the format of this object. That is, the amount of memory to be allocated for it, the amount of slots, and if they are pointers or bytes or what. The format is known by the definition of undefined object.

Fortunately, nil has no pointers to other objects, except for his class, simplifying the process. We will set nil’s class once we create it later.

We create the classes

Creating a class is a complex operation. A class has a metaclass. And it has a name which is a symbol (unique in the system). And it has a superclass, which may have not been created yet. And it has a dictionary of class variables. And… a lot of stuff.

Our objective is to keep this bootstrap the simplest. And for that, we will delay all the complex operations to the moment when they are not so complex. In this step we create empty classes. We only initialize their format with a SmallInteger, and we let the rest of their pointers pointing to nil.

The first step for creating a class, is to create its metaclass. And to create a metaclass, we need the class Metaclass. This Metaclass, in a ST-80 like model, follows the Metaclass<->Metaclass class loop. That is, Metaclass is an instance of Metaclass class, and Metaclass class is an instance of Metaclass, as shown in blue in the following figure.

SmalltalkMetaclassesWe create the first Metaclass and Metaclass class as objects without class, and then we make each one an instance of the other.

metaclassMirror := objectSpace
    createClassWithFormat: classFormat
    forInstancesOfFormat: metaclassFormat.
metaclassClassMirror := objectSpace
    createClassWithFormat: metaclassFormat
    forInstancesOfFormat: classFormat.

metaclassMirror 		setClass: metaclassClassMirror.
metaclassClassMirror 	setClass: metaclassMirror.

Once we have the first metaclass, we can create all the classes.

self behaviorDefinitions do: [ :aClassDefinition |
	| newClass newClassMetaclassMirror theMetaclassMirror |
	theMetaclassMirror := objectSpace classNamed: #PCMetaclass.
	newMetaclassMirror := theMetaclassMirror basicNew asClassMirror.
	newMetaclassMirror format: aClassDefinition classSide format.

	newClass := newClassMetaclassMirror basicNew asClassMirror.
	newClass format: aClassDefinition format.
	]
]

At this point, the classes only have set their format, and their class. All other slots have pointers to the nil object we created at the beginning.

Fix nil, create true and false!

Now we created all the classes, even if they are empty, we can fix the “classless” nil and create our true and false objects.

theNil setClass: (objectSpace classNamed: #PCUndefinedObject).
theTrue := (objectSpace classNamed: #PCTrue) basicNew.
theFalse := (objectSpace classNamed: #PCFalse) basicNew.

Initialize the classes state

Now we have all classes created, and the three basic objects we need (nil, true and false). So now we can start initializing all our classes. This initialization consists for each class in:

  • Set the superclass of the class. The root of the hierarchy should be nil.
  • classDefinition superclass isEndOfHierarchy ifFalse: [
        superclassMirror := objectSpace
            classNamed: classDefinition superclass name.
    ] ifTrue: [
        superclassMirror := objectSpace nilObject.
    ].
    classMirror superclass:superclassMirror.
    
  • Set the class name
  • classMirror className: classDefinition name.
    
  • Set the collection of instance variables of the class
  • classMirror instanceVariables: classDefinition instanceVariables.
    
  • Set the superclass of the metaclass. The root of the hierarchy should be PCClass.
  • metaclassMirror := classMirror classSide.
    classDefinition superclass isEndOfHierarchy ifFalse: [
        metaclassMirror superclass: superclassMirror classSide.
    ] ifTrue: [
        metaclassMirror superclass: (objectSpace classNamed: #PCClass).
    ].
    
  • Set the instance side relationship of the metaclass
  • metaclassMirror := classMirror classSide.
    metaclassMirror
        instanceSideClass: classMirror.
    
  • Set the collection of instance variables of the metaclass
  • metaclassMirror := classMirror classSide.
    metaclassMirror
        instanceVariables: classDefinition classSide instanceVariables.
    

After this initial initialization is performed for every class, we finish by initializing the class variables. Class variables are represented by a Dictionary object. A dictionary object is an object with a complex structure, and the way to manipulate it depends on the nature of the system we are bootstrapping. The solution, so far, is to delegate the initialization of the dictionary to the dictionary itself.

For that we use a combination of the code of the dictionary and an AST interpreter. An AST interpreter needs to be initialized before its usage so later, all class variables can be initialized. As you can see in the code below, the AST interpreter usage is hidden inside the mirror implementation :).

objectSpace initializeInterpreterForCodeProvider: self kernelSpec.
self behaviorDefinitions do: [ :classDefinition |
    | classMirror |
    classMirror := objectSpace
        classNamed: classDefinition name.
    classMirror
        classVariables: classDefinition classVariables.
].

Install methods

Now we have all classes of the new system created and initialized. We can start installing all their methods. Before, we should declare all global variables of the system, so the compiler knows how to bind them correctly. After that, we take the source code of all methods from the system description and compile them. The compilation gives us as result the bytecode of the method + the literals. This method is then translated to a method in the new world, and installed into the new system.

The globals initialization looks like:

objectSpace environment
            addGlobal: #Processor
            pointingTo: objectSpace nilObject.

Then, for each class, we have the following code to create and install the methods:

"build the methods as instances of this system"
newMethods := aMethodBuilder
    methodsForBehavior: mirror
    fromDefinition: aBehaviorDefinition.

"create a method dictionary of the new system"
newMethodDict := objectSpace createMethodDictionary: newMethods size.
newMethods do: [ :m |
    "install a method from this system to the other"
    "the translation to a method to the other side is made inside"
    newMethodDict installMethod: m
].
"we set the method dictionary to our class"
mirror methodDictionary: newMethodDict.

Initialize the system state

Finally, we initialize the system state, with the aid of the AST interpreter. This last step consists mainly in:

  • execute the initialize class side methods
  • set up the process scheduler of the system and install its processes
objectSpace interpreter evaluateCode: 'PCCharacter initialize'.
objectSpace interpreter evaluateCode: 'PCString initialize'.
objectSpace interpreter evaluateCode: 'PCFloat initialize'.

objectSpace interpreter evaluateCode: '
    Processor := PCProcessorScheduler basicNew.
    Processor initProcessLists.'.

process := objectSpace
        createProcessWithPriority: 3
        doing: 'PCSystem start'.
objectSpace installAsActiveProcess: process.

Conclusion

Bootstrap: achieved.

With our new pieces into the game, we were able to overcome the virtual machine limitations, and have a in-image full bootstrap. The next steps go in the way to:

  • serialize this object graph into an image file, so it becomes autonomous
  • test the bootstrapped system while still living along with the original system, without serializing it. I mean, run it into the same VM without AST interpreter.

I hope I explained myself well.
Arrivederci!!
Guille

Keymappings 101 – for Pharo 2.0

Pharo

Pharo 2.0 release includes the Keymappings library. Keymappings is a library for configuring shortcuts for the current UI library (Morphic). It models concepts like: shortcuts, key combinations, event bubbling. It is a very simple library which I’ll introduce gradually in this post.

Key combinations

Keymappings main task is it’s ability to associate a key combination to an action. So we have to build up those key combinations. The simplest key combination is the one that gets activated when a single key is pressed. We call these combinations single key combinations:

$a asKeyCombination. -> "single key combination for A key."
Character cr asKeyCombination. -> "single key combination for  key."

Although, usually key combinations get a bit more complex. It is very common to combine single keys with meta keys or modifiers. These meta keys or modifiers are the well known ctrl, shift, alt and command keys. To build a modified key combination we can do as follows:

$a ctrl. -> "a modified key combination for Ctrl+A"
$a ctrl shift. -> "a modified key combination for Ctrl+Shift+A"
It is important to notice that all key combinations are not case sensitive. It takes a and A characters as the same, since they are the same key.

Have you ever used emacs, Eclipse or Visual Studio? Then you probably know sequences of key combinations that launch one only action. Like Alt+Shift+X, T (to run JUnit tests in eclipse)? So keymappings can do that too:

$a command shift, $b shift. -> "key sequence (Cmd+Shift+A, Shift+B)"

Sometimes, you want to configure an action to be activated in two different cases. Those are Keymapping options, and get activated when one of the options gets activated:

$a command | $b command. -> "key combination (Cmd+A or Cmd+B)"

Finally, since Pharo is a cross platform system and it is important to provide a good user experience by with the most suitable shortcut layout, keymapping implements platform specific shortcuts, which get activated only when running in the specific platform:

$a command win | $b command unix. -> "Cmd+A on windows, but Cmd+B on unix"

Shortcut configurations

Now you know how to build key combinations for your purposes, you probably want to go to the action. Map those combinations to actions and make them work!

Single shortcut configuration

The simplest way to attach a shortcut to a morph is by sending him the #on:do: message. The first argument expected is a key combination and the second one is an action. In the example below, a workspace is created with two shortcuts:

  • when Cmd+Shift+A is pressed, the workspace is deleted
  • when Cmd+Shift+D is pressed, an information growl should appear yelling ‘this shortcut works!’
w:= Workspace new.
morph := w openLabel: 'keymapping test'.
morph on: $a shift command do: [ morph delete ].
morph on: $d shift command do: [ UIManager default inform: 'this shortcut works!' ].

Easy, huh? So let’s move on…

Shortcut categories

Sometimes you want to group and organize shortcuts in a meaningful way and apply them all together on a morph. Sometimes you want some morphs from different hierarchies to share the same group of shortcuts easily. Those groups of shortcuts are what keymapping calls Categories. A category is a group of shortcuts, so far (will change in the future) defined statically by using a keymap pragma on class side:

"defining a category"
SystemWindow class>>buildShortcutsOn: aBuilder
    <keymap>

A class side method marked as <keymap> will be called with a builder object, which can be used to define a named set of shortcuts:

SystemWindow class>>buildShortcutsOn: aBuilder
    <keymap>
    (aBuilder shortcut: #close)
        category: #WindowShortcuts
        default: $w ctrl | $w command mac
        do: [ :target | target delete ]
        description: 'Close this window'.

Shortcuts defined through the builder specify the name of the category they belong to, a default key combination, an action, and a description. All this metadata is there to be used as settings in the future.

Finally in order to get your morph handle those shortcuts you can use the #attachKeymapCategory: message as in:

w:= Workspace new.
morph := w openLabel: 'keymapping test'.
morph attachKeymapCategory: #Growling.

Bubbling

Keymappings’ shortcuts bubble to their parent if not handled, up until the main world morph. That has two main consequences:

  • Shortcuts for your application can be designed in a hierarchical way and;
  • Every time a shortcut does not work for you, it means that a morph below you has handled it ;) (be careful with text editors that handle loooots of key combinations)

Future work

So far, so good, but there is some plan on Keymappings for Pharo 3.0 development, which I can anticipate:

  • Some API changes: #on:do: can be confused with exception or announcement handling. #asShortcut will probably be properly renamed as #asKeyCombination. There is an inconsistency between the #command and #ctrl messages…
  • A lot of renames and new comments :)
  • Spread it all over the system
  • Make keymap categories first class objects, not any more a symbol ;)

à la prochain!
Guille

Pharo 2.0 Released

Pharo Project

Aaaand, a new version of the Pharo project cames out. It ressembles the version 2.0 of this dynamically typed object-oriented programming language and environment. This release includes many cool stuff improving a lot the infrastructure of the system, adding new core libraries and lots of cleansing and improvements. Let’s make some remarks on this release.

Cool Development Tools, all by default

Pharo’s default browser is now Nautilus by Benjamin Van Ryseghem. Nautilus has lots of cool features, like an alternative Group view, a plugin architecture, and integration with Monticello, refactorings and the Critics browser. Yes! Now by default Pharo includes refactorings, since they are one of the cornerstones of the development activities. Critics browser is also included, so the code quality can only improve :).

Auto completion has also seen lots of changes: default completion is <enter>, press <tab> to complete word per word á la command line, and it has also been revisited to provide better and more meaningful results.

Finally, if you press <shift+enter> you’ll see on the right upper corner the Spotlight by Esteban Lorenzano. A simple but powerful way tool to quickly browse a class or method.

Boosted by NativeBoost and Fuel

Pharo wants to be fast. And that’s something NativeBoost and Fuel achieve. That’s why you can find them included by default in the system. NativeBoost (by Igor Stasenko) gives us the ability to execute machine code from the language side, and a new generation FFI with callbacks. Use it with caution :). Fuel, written by Mariano Martinez Peck and Martin Dias, is a cool object serializer focusing on fast deserialization (materialization), and the ability to serialize any kind of objects: Block closures? yes. Contexts? yes. Complete debuggers so we can restore them and debug failures in other environments? YES.

UI Front – Spec and Keymappings

Pharo 2.0 includes two new cool libraries on the UI front: Spec and Keymappings.

Spec is a framework, mainly work of Benjamin Van Ryseghem under the tutelage of Stéphane Ducasse, to build UI components declaratively. It puts its main focus on component reuse and ability to be composed. Spec was included into Pharo 2.0 and some tools were reimplemented to use it. How do you give it a try?

On the other side, Keymappings is a shortcut library mostly re-written by me (Guille Polito) to adapt it to Pharo. It’s main objective is to provide common shortcut semantics for desktop UIs, and remove hardcoded semantics spread all over the system. Pharo 2.0 includes Keymappings and has already replaced some users of the old-fashioned(harwiredandmessy cof cof) shortcut declaration by nice keymapping ones. On the documentation side, I owe it to you :). I promise to a nice tutorial-post this week!

System changes – System Announcements, RPackage, FileSystem, branded VM

On the internals of the system, the notification of system events was replaced by System Announcements, RPackage was introduced so the old and ugly packaging system can be slowly migrated, and the old FileDirectory was tackled down and all its usages were replaced by the new cool FileSystem library (already there in 1.4).

Also, the Pharo VM is now branded, and includes many fixes and bundled libraries (nativeBoost and SSL plugins, cairo, freetype). You should run your Pharo images on a Pharo VM, which you will identify by a nice Pharo icon ;).

And of course there are lots of other clean and cool new stuff to see like SSL, command line tools, non UI blocking notifications… A more detailed list is here. So take Pharo, have a look, enjoy, and give feedback. Remember that any contribution is valuable, as small as it looks.

Download Pharo
Pharo website
Joining and helping
Pharo By Example book (available as a free PDF)
Screencasts!
Reporting problems
Pharo vision document

Chaus, Guille

The Bootstrap Chronicles – The end of a phase

As some of you already know, the GSOC project edition 2012 is coming to an end.  And along with it, the bootstrap project reaches a checkpoint. This post covers is the news since the last chapter, and discusses about the future steps. In a next post I’ll document  the details of the project deliverables.

Where we where?

The first product of this project was a renewal of an image serializer: the SystemTracer. It takes an object graph and serializes it into the image format a vm works with. The System tracer was refactored reifying the memory object formats and updated to write Cog images.

The second step was to work on bootstrapping. And it was successful.  Hazelnut, the bootstrap process tool, is able to build a smalltalk image from a description. To ensure the quality and health of these newly created images I set up jenkins jobs loading different packages on top of them, and running tests over them.

What happened since last time: Declarative kernel descriptions

A kernel definition has two main parts:

  • the code and definitions of the entities of that kernel
  • a definition on how to build the basic model of that kernel and how to initialize it finally

The first one can take the form of source files like in https://github.com/guillep/PharoKernel, which is the source code Hazelnut actually uses to bootstrap Pharo images. The second part of the kernel definition contains some imperative parts, and by now they are declared in simple pharo classes you download from Monticello.

So now, the bootstrap loads the kernel definition from those source files, generates the bootstrapped environment, and serializes it into a new image file.

The future of bootstrapping pharo

Since our goal is to bootstrap pharo to support it’s modularity and  evolution, there are some keypoints to attack in the near future:

  • getting the pharo sourcecode in sync with this bootstrap representation
  • choosing the really important parts for a kernel. What should be and what should not in those source files? Where do we package what’s not going kernel?
  • building pharo from bootstrapped images.

Even, when looking at the upcoming pharo changes like first class slots and class layouts, or the new Tanker package manager, the bootstrap will need for sure some updates.

Conclusions

I hope this project makes pharo grow and get better! We can now generate images with the source code defined statically in source files, so for the GSOC program the scope has been fulfilled.

See you in a next post documenting the project!

Salut!