Tag Archives: smalltalk environment

Oz object spaces in Esug 2013

Hi!

This little post is to add a summary of what I’ll be showing in the upcoming ESUG conference. I’ll present a paper about my work in the workshop and present the recovery tools of Oz in the innovation awards :).

For the paper

I’ll present the ideas and implementation of Oz object spaces. Especially the metacircular problems it solves, and a bit on the vision we (I and my supervisors have on this line of work). If you want to read the paper, you can go to:

http://rmod.lille.inria.fr/archives/papers/Poli13a-IWST13-ObjectSpacesVirtualization.pdf.

For the lazy, I paste here the abstract:

Reflective architectures are a powerful solution for code browsing, debugging or in-language process handling. However, these reflective architectures show some limitations in edge cases of self-modification and self-monitoring. Modifying the modifier process or monitoring the monitor process in a reflective system alters the system itself, leading to the impossibility to perform some of those tasks properly. In this paper we analyze the problems of reflective architectures in the context of image based object-oriented languages and solve them by providing a first-class representation of an image: a virtualized image.
We present Oz, our virtual image solution. In Oz, a virtual image is represented by an object space. Through an object space, an image can manipulate the internal structure and control the execution of other images. An Oz object space allows one to introspect and modify execution information such as processes, contexts, existing classes and objects. We show how Oz solves the edge cases of reflective architectures by adding a third participant, and thus, removing the selfmodification and self-observation constraints.

For the innovation awards

I’ve prepared a demo and with that I recorded a little video showing it. You can see it in:

Also, we have a sexy logo!

Oz logo

Oz logo, created by my friend Ximena Fernandez
Copyright (C) 2013 Ximena Fernandez

I asked my friend Ximena to make a logo for my projects, even pushing her with the deadline of ESUG 2013 (sorry xime ;), and she created this one, that I really like.

So, thanks again Xime!

Now, waiting for next week @ ESUG, to taste the ESUG beers ;).
Enjoy!
Guille

The Bootstrap Chronicles – The end of a phase

As some of you already know, the GSOC project edition 2012 is coming to an end.  And along with it, the bootstrap project reaches a checkpoint. This post covers is the news since the last chapter, and discusses about the future steps. In a next post I’ll document  the details of the project deliverables.

Where we where?

The first product of this project was a renewal of an image serializer: the SystemTracer. It takes an object graph and serializes it into the image format a vm works with. The System tracer was refactored reifying the memory object formats and updated to write Cog images.

The second step was to work on bootstrapping. And it was successful.  Hazelnut, the bootstrap process tool, is able to build a smalltalk image from a description. To ensure the quality and health of these newly created images I set up jenkins jobs loading different packages on top of them, and running tests over them.

What happened since last time: Declarative kernel descriptions

A kernel definition has two main parts:

  • the code and definitions of the entities of that kernel
  • a definition on how to build the basic model of that kernel and how to initialize it finally

The first one can take the form of source files like in https://github.com/guillep/PharoKernel, which is the source code Hazelnut actually uses to bootstrap Pharo images. The second part of the kernel definition contains some imperative parts, and by now they are declared in simple pharo classes you download from Monticello.

So now, the bootstrap loads the kernel definition from those source files, generates the bootstrapped environment, and serializes it into a new image file.

The future of bootstrapping pharo

Since our goal is to bootstrap pharo to support it’s modularity and  evolution, there are some keypoints to attack in the near future:

  • getting the pharo sourcecode in sync with this bootstrap representation
  • choosing the really important parts for a kernel. What should be and what should not in those source files? Where do we package what’s not going kernel?
  • building pharo from bootstrapped images.

Even, when looking at the upcoming pharo changes like first class slots and class layouts, or the new Tanker package manager, the bootstrap will need for sure some updates.

Conclusions

I hope this project makes pharo grow and get better! We can now generate images with the source code defined statically in source files, so for the GSOC program the scope has been fulfilled.

See you in a next post documenting the project!

Salut!

Smalltalk behind the scenes: the meta model

Have you ever evaluated this pieces of code in Pharo?

 ProtoObject superclass.
 ProtoObject class superclass.
 Metaclass class class = Metaclass.

Wait, WTF? How is that ProtoObject superclass is nil? Wait again, and the one of it’s superclass is Class? Metaclass class is an instance of Metaclass? Hey, that’s kind of the chiken and the egg problem, which one was first?

You know that when you create a class, you specify a superclass for it.  This superclass will specify some other properties and the VM will use it to perform the method lookup.

Also, probably you already know that when a class is created in Smalltalk, a metaclass is created for it implicitly. That metaclass describes the class side behavior: class side methods, class instance variables…

Funny thing about this implicit metamodel, is that a second class hierarchy is built in parallel to the original class hierarchy.

Now, if you think about this, you can understand why the method lookup works also in the class side methods, and they are not static like in Java or C# :).

You can have a look at the following invariants:

aClass superclass class = aClass class superclass.
aClass class class = Metaclass.

Which of course have it’s exceptions. The method lookup ends when it reaches a class whose superclass is nil.  And the class side objects also behave like a Class, because they finally inherit from Class. HA! But then the metaclass hierarchy re-enters the non-metaclass hierarchy. Thinking of this in an operational way is kind of meta confusing, isn’t it?

But this is not the motivation of this post. The motivation is this: Are we really coupled to that meta model?  How can I create my own?

If you remember from my post on vm limitations I learned during the bootstap, the vm only expects 3 things from a class:

  • that it’s first instance variable is it’s superclass.
  • that it’s second instance variable is it’s method dictionary.
  • that it’s third instance variable is it’s format.

Any object respecting that contract can be treated like a class by the VM.  Then you can think on creating your own metaclass loop, kind of independent from the original one…

classFormat := ...
metaclassFormat := ...

"This metaclass defines how our metaclass instances will be.  It is only here to define the first metaclass format, and it will be discarded"
metaclassClass := Metaclass new.
metaclassClass
superclass: Class
methodDictionary: (MethodDictionary new)
         format: classFormat.

metaclass := metaclassClass basicNew.
metaclass instVarAt: 1 put: Metaclass.
metaclass instVarAt: 2 put: MethodDictionary new.
metaclass instVarAt: 3 put: metaclassFormat.

metaclassClass := metaclass basicNew.
metaclassClass instVarAt: 1 put: Metaclass class.
metaclassClass instVarAt: 2 put: MethodDictionary new.
metaclassClass instVarAt: 3 put: classFormat.

metaclassClass adoptInstance: metaclass.

Once you have a metaclass, instantiate it to create your class, and instanciate it to create your little object! That’s crafting Smalltalk using Smalltalk. Well, that is bootstrapping the meta model :).
The only ugly thing is that in order to create a new meta model with different instance variables, you have to create a transient class in the middle, because the VM does not like to have objects with a format X, whose class defines a format Y… So the hack just solves the format problem :).

Now you can think about simpler stuff like a class instance of itself, subclass of nil. Or more complex one :).

You can change it, I told you. Now it’s up to you how to use it…

The Bootstrap Chronicles Chapter 3 – It’s Alive!

So now that you now a bit what the bootstrap is about and what are some of the problems to face.  I’ll show you some solutions and progress for real.

How does the bootstrap implementation work

To bootstrap a Smalltalk environment, you need to create a new environment, with it’s classes and objects, and initialize some state in them. I could have done that in C, using mallocs and initializing everything by hand using plain memory :). But doing it in smalltalk is easier: you have late binding, polymorphism, closures…  Even, once you have done your first steps in the bootstrap, you can send messages to your objects. THAT is nice.

So, that is the way we cho0se to go: A new environment is created (a guest) into the current environment (the host).  The guest will have it’s own classes and objects.

What about the special objects of the vm?  We share them with the host environment, because if not, out new image will not be able to run… Afterwards, when this new image is written into an image file, we will swap references to point to our own new special objects array.

Here is a picture of how it looks like:

How the host and guest are related

The Current Version

I’ve been through several versions of the image with different capabilities, sizes, correctness. You have to know, everything you forgot to initialize, or initialize in a wrong order, or if you took an extra object you do not need, you will have a not running image, or one that carries all the objects in the host also…

Another thing is that current version takes a sample of the objects in the host to build the guest. I’m already working on starting from scratch + source code, but that is the future and I like enjoying the present :).

So, how do you load the current code?

Gofer it
    url: 'http://www.smalltalkhub.com/mc/Guille/Seed/main';
    package: 'ConfigurationOfHazelnut';
    load.

(ConfigurationOfHazelnut project version: '1.3') load.

Also, as the code is very sensitive on what you do,  it is also sensitive on what image you’re running it on.  If you play with it in a wrong/different image, you will have different/unexpected results.  So, I suggest you to use the same image as me for testing it: Latest Pharo 2.0.  In particular, I’ve tested it on versions 20133 and 20134.

Do not scare when loading the configuration on Pharo 2.0 for the first time It will raise an error. It is an issue related with unzipping old mcz in Pharo (http://code.google.com/p/pharo/issues/detail?id=6054) which will fortunately fixed soon. Just close the debugger, try again, and get the project working.

And, How do I try these weird stuff?

With Nicolas Petton, we have written some examples in the class named HazelBuilderExamples.

To run them, you can try the following scripts:

"Writes an image which when opened prints a spaceTally on fileok.txt"
HazelBuilderExample new buildImageWithSpaceTally

"Writes an image which when opened prints a report on all the packaging/initialization deficiencies of the image on fileok.txt"
HazelBuilderExample new buildImageWithBrokenReferencesReport

"Writes an image which when opened prints a report on all the packaging/initialization deficiencies of the image on fileok.txt. The report is written in the xml format Jenkins Junit plugin likes."
HazelBuilderExample new buildImageWithBrokenReferencesReportForJenkins

After evaluating this code, you’ll have some bootstrapped image and changes files. Open that file with your CogVM and wait until it closes.  Then have a look at the fileok.txt file :).

Of couse you can look at the code in the examples, and try to build your own. Email me if you have ideas to improve this :).

There are also probably some problems with file overwritting that came with some latest Pharo changes with the file management.  Please, if you notice this, just remove the bootstrapped* files from the folder where your image is and try again.

Hey! How is this thingy useful?

Well, if you’ve had a look at the examples, the three examples I’ve shown you are very useful:

  • The first one tells you how the space is distributed in the new image. You can use this knowledge to attack space problems if you want an even smaller image.
  • The second one is used to detect bugs in the pharo Packaging or in the initialization process: You have not initialized some classes, or the have been initialized but the initialization code does not initialize all the variables.
  • The third one is the same as the second, but adapted to have this kind of CI integration: http://car.mines-douai.fr/ci/job/Seed%20Broken%20References%20Report/

Nice huh? Now we can use jenkins to validate the core of Pharo is well initialized and well packaged when that list becomes empty.

What’s next?

  • Each one of those things in the list should be tracked as issues.
  • continue working on the bootstrap from sourcecode.
  • Making the fuel seed work so we can install fuel packages on our little image.

Keep u updated!

Hasta Luego!

Guille