Bistro = Smalltalk over JavaTM

Nik Boyd

Copyright 1999 Nikolas S. Boyd.
Permission is granted to copy this document provided this copyright statement is retained in all copies.


Introduction

Bistro is a new programming language that integrates the best features of Smalltalk and Java. Bistro is a variation of Smalltalk that runs on top of any Java virtual machine (VM) that conforms to Sun's Java specifications. Bistro offers Smalltalk developers a means by which to reuse the models they built with Smalltalk. Bistro provides mechanisms for translating Smalltalk model code into Java, so that models originally built with Smalltalk may be deployed on and execute in a Java environment. Further, like Smalltalk, Bistro offers software developers the ability to continue to build software models with a syntax that approaches the expressiveness and readability of natural language.

This paper presumes familiarity with both Smalltalk and Java. It focuses primarily on the issues associated with integrating the two languages. The paper is organized as follows. First, the motivations that led to the development of Bistro are discussed, related work is cited, and the goals for the Bistro project are outlined. Next, an overview of the Bistro language is provided and then many of the design decisions that permit Bistro to interoperate in the Java world are discussed. Finally, the current status of Bistro and plans for its further development are provided.

Also, please see the Bistro project page. The Bistro compiler and Smalltalk classes have been released!


Background and Motivations

Over the past 25 years, Smalltalk pioneered and spawned several advanced computing technologies that have progressively become popular and then prevalent within the computing industry. Technologies pioneered by Smalltalk include the mouse, high resolution bit-mapped graphics displays, graphical user interfaces with overlapping windows, object-oriented programming, and virtual machines with automatic garbage collection.

Smalltalk has a long-standing history as an open source environment. So, commercial Smalltalk vendors historically derived most of the monetary value of their licenses from proprietary virtual machine implementations. However, over the past few years, Java VMs have been adopted throughout the computer, communications and consumer electronics industries, and adoption continues unabated. Some vendors have gone so far as to integrate a Java VM into their operating systems. With such wide-spread availability of Java VMs, the question naturally arises, why buy a proprietary VM when they are standardized (de facto) and readily and freely available? This question was one of the factors that contributed to the idea of running Smalltalk on a Java VM. Overall, Java offers many benefits to software developers:

  • Object-orientation
  • First-class interfaces
  • Strong typing
  • Exception handling
  • Name spaces
  • Multithreading
  • Garbage collection
  • Freely available virtual machinery
  • Web and internet connectivity
  • Event-driven graphical user interfaces
  • Client-server architecture
  • Deliverable behavior and content
  • Strong and fine-grained security

So, one might well ask, what's missing? Why bother porting Smalltalk to Java? Ultimately, it's about modeling concepts in software. Language expressiveness and readability are still the greatest advantages Smalltalk has over Java. While this advantage may be lost on programmers responsible for lower level system and infrastructure facilities, it fulfills many needs of application programmers, especially those responsible for modeling business enterprises - e.g., with business objects.

Also, the essential Smalltalk classes are well designed and mature. Many design ideas from Smalltalk have been incorporated into Java's classes. However, Java's classes still have some neglected, immature and inflexible areas. Important methods are missing from some of the simple Java classes like String and Point. Some important Java classes have been made final unnecessarily. This prevents further derivations to extend their behavior and imposes a design burden on developers. Developers must often weaken their object-oriented designs by finding odd places to put missing behavior that, in a good object-oriented design, belong in an existing base or an extension class.

Agility is another advantage Smalltalk has over Java. The absence of type information in Smalltalk contributes to its agility during software development. Software prototyping is much easier without type information. Specifying type information pervasively throughout a software system during initial development requires much more time of the developer. And, the type information simply makes the resolution of method implementations safer and more efficient. So, the best option is to have a spectrum of possibilities for improving performance when and where needed.

For these reasons, Bistro provides such a spectrum of options. Smalltalk developers will feel at home with the ability to write code with familiar idioms and classes. Then, as application interfaces harden and classes mature, type information may be added to the classes in order to improve the performance of method resolution when and where needed. Performance critical methods may even be recoded directly in primitive Java methods. Finally, if 100% purity of the Java code is not an issue, a Java native interface (JNI) can be used to create platform specific optimizations written in C or C++.

In 1996, Jeff Sutherland warned the Smalltalk community that Java posed a significant threat to the Smalltalk market. In his paper, The Smalltalk Manifesto,1 he argued that Smalltalk must "Get with the Net" to remain relevant. Consider how Bistro addresses each of the following points he made:

  1. Smalltalk applets must distribute over the net.
Bistro classes are translated into Java class files, which may then be distributed over the net to servers or clients.
  1. Smalltalk must be free.
The Bistro project will be conducted as an open source project once an adequate compiler has been built.
  1. Smalltalk must be standard.
NCITS adopted the work of X3J20 in December, 1998.
  1. Smalltalk development tools must be used to support Java development.
Bistro seamlessly integrates Smalltalk with Java. This point will be answered fully once the Smalltalk tool set has been ported - i.e., once Bistro has integrated a browser, a compiler and a debugger.

Now, three years after his Manifesto, Sutherland's estimation of the motivations of the Smalltalk vendors seems woefully naive. Far from enthusiastic response to and fulfillment of his challenge, many of the leading Smalltalk vendors (IBM, GemStone, ObjectShare, Smalltalk Systems, ...) have embraced Java to some significant degree. While this is clearly a good thing (both for the vendors and for Java), success in the Java market must certainly have demotivated them from pursuing the agenda Sutherland outlined. Also, while many of the Smalltalk vendors have released free versions of their products, they still require license fees for versions intended for commercial use. One possible explanation is that they are still recovering the investments they originally made in developing their proprietary Smalltalk VMs.

Also, Sutherland's estimation of the ease of porting Smalltalk to Java were optimistic. Even when Java provided the features necessary to support language integration (e.g., inner classes), a complete mapping is far from trivial. This paper explores many of these issues in detail and offers (at least tentative) solutions to the problems raised.


Related Work

Traditionally, Smalltalk systems are built in the context of an object memory image. While an image based development environment contributes significantly to the agility of Smalltalk's integrated suite of tools, it introduces some difficulties for code, component and system configuration management in large development projects. A declarative, file based programming model makes it much easier to use existing version control and configuration management tools. Some of the newer Java development tools have incorporated incremental compilation facilities that make Java programming almost as agile as Smalltalk. So, it's certainly feasible to have rapid development with a file based language.

The idea of having a declarative language model for Smalltalk has been explored previously. The strongest proponent for a declarative Smalltalk syntax has been Allen Wirfs-Brock. His years of work2 finally achieved fruition,3 but has not as yet found its way into a commercially available Smalltalk product. However, it did have a significant influence on the standards work of the ANSI X3J20 workgroup.4

Initial discussions regarding the integration of Smalltalk and Java began as early as 1996 with Per Bothner.5 Bothner considered the differences between Smalltalk and Java base classes (like Object), and several mechanisms for implementing Smalltalk method lookup and dispatch using the Java mechanisms then available.

Shortly thereafter, in 1997, a pair of language comparisons between Smalltalk and Java were published,6, 7 and several more approaches to integrating Smalltalk and Java were explored. Nik Boyd8 explored mechanisms for organizing Smalltalk classes into packages, mapping Smalltalk classes and methods to Java classes and methods, mapping Smalltalk blocks to inner classes, and method interoperability and combination.

In 1997, Andrew Eidsness created Jasper9 as part of his Honours Project for Carleton University. Eidsness also proposes mechanisms for generating Java class files from Smalltalk. Some aspects of Eidsness' approach parallel those found in Bistro - e.g., method selector renaming for operators and an implementation of first-class metaclasses. However, there are also several differences, including how blocks are implemented, as well as dynamic method dispatch. Eidsness' focuses on compiling Smalltalk code verbatim, rather than evolving the language as does Bistro.

1997 also saw Claus Gittinger and Stefan Vogel built support for Java into the Smalltalk/X virtual machine.10 This approach reverses that proposed in this paper. Rather than compiling Smalltalk into byte code for a Java VM, Java is compiled into byte code for a Smalltalk VM.

1998 saw the advent of Smalltalk to Java conversion services. Consultants at both Orisa and GEBIT in Germany offer migration services utilizing proprietary conversion tools.11, 12

Bistro is founded on previous research regarding Smalltalk and Java integration. Initially, the basic feasibility of integrating the two languages was explored. Having established the feasibility of integrating the two languages, the possibility of adding type information to Smalltalk to achieve full integration with Java was explored. This resulted in the design of the Jist programming language.13 However, requiring pervasive type information significantly complicated the resulting language. Further experimentation with the grammar found a means for making the type information optional. The result is Bistro, a declarative variation of Smalltalk with optional typing whose class files run on the Java VM.


Goals

The primary goals of Bistro include the following:

  1. Maintain the simplicity, expressiveness and readability of Smalltalk.
  2. Support the ANSI standard Smalltalk behaviors.4
  3. Support seamless integration with existing Java classes.
  4. Provide mechanisms for porting Smalltalk code to a Java runtime environment.
  5. Optimize untyped messages as much as technically feasible without modifications to the Java VM.
  6. Provide a natural performance optimization path with the addition of type annotations and primitive methods written in Java.

As a side benefit, Bistro provides a natural migration path from Smalltalk to Java, for those software developers who want to commit their models to Java entirely. However, the primary focus of Bistro is to support continued development of software models using a higher-level language model that approaches the expressiveness and readability of natural language.

The level of integration and immediacy of the Smalltalk tools support very high productivity and rapid prototyping. So, a final goal is to eventually have a development environment as integrated and agile as that traditionally found in Smalltalk, but hosted on the Java VM.


Smalltalk over Java: An Overview of Bistro

A bistro is a place in France where people meet to share coffee and small talk.
-- Jean-François Cloutier, Co-Founder, Noonetics

Bistro represents a further evolution of Smalltalk, along with its integration with Java. Syntactically, it resembles Smalltalk more than Java. Although, there are several language features from Java that have been incorporated into Bistro to support the integration with Java. Table 1 shows which language elements and features from Smalltalk and Java are combined in Bistro.

Table 1. Bistro Language Features
Smalltalk Java
Literals self, super, nil, true, false, numbers, strings, symbols, arrays
Messages unary, binary, keyword: (:)
Punctuation "." for statements
";" for cascades
[ ] for Smalltalk methods and scopes
Operators standard operators: , @ + - * / \ < = > ~ & |
and their combinations: == <= >= ~= ~~ // \\
Metaclasses and metapolymorphism, i.e., inheritance of metabehavior
Access Controls public, protected, private methods and classes
Derivation Controls abstract, final methods and classes
Thread Control synchronized methods
JNI Support native methods
Primitive Methods { } for Java methods and scopes
Packaging packages, imports,
Types classes, interfaces, and optional typing on variables and arguments
Variables inline variable declaration and initialization

Bistro also extends Smalltalk syntax to support anonymous keywords (:) in messages and message patterns. This feature is similar to the way that block arguments are declared in Smalltalk. It provides a direct means by which a Smalltalk method may invoke a method with multiple arguments in an existing Java class. For further discussion of this, see the sections on keyword messages and method interoperability.

Java uses ";" as a statement terminator. Smalltalk uses "." as a statement terminator and ";" for message cascades. The Smalltalk syntax more closely approximates the punctuation used in natural language, especially English. Also, message cascades are frequently used in Smalltalk to simplify sending a series of messages to a single object. For these reasons, Bistro uses the Smalltalk statement punctuators.

All the Java reserved words used in Bistro are adjectives, which are seldom (if ever) used for Smalltalk variable or method names. All of these words duplicate their equivalent semantics from Java - i.e., they are translated verbatim into Java. However, in Bistro, class and interface are not reserved words, because they are nouns and so may be needed as identifiers. Where needed during identifier translation, the Bistro compiler adds a prefix ($) to words that are reserved in Java, so that Java compiler errors are avoided.


Access Controls

Access controls play an important role in defining strong interaction contracts in object-oriented designs. Like Java, Bistro provides strong access controls on class members - methods and variables. However, the defaults for Bistro are different from those found in Java. They duplicate those found in Smalltalk, which has no explicit access controls. Table 2 shows a comparison of the default access controls between Java and Bistro.

Table 2. Default Access Comparison
Definition Java Bistro
class packaged * public
metaclass n/a public
type (interface) packaged public
metatype n/a public
variable packaged protected (for both class and metaclass variables)
method packaged public (for both class and metaclass methods)
* Packaged access is the standard default access for Java - i.e., accessible to classes defined within the package.

As in Java, control over access can be specified in member declarations. Because Bistro runs in the Java VM, these access controls are enforced at runtime, making them much stronger. Thus, like Java, Bistro supports the design of strong contracts between objects.


Declarative Language Model

In Smalltalk, class definitions have traditionally been interpreted in the context of a presupplied object memory image. But, declarative language models provide distinct benefits for the control of source code and the management of system construction and configuration. So, class structure is another aspect of Java incorporated into Bistro. Thus, each class has both a source file (ClassName.bist) and a binary file (ClassName.class). While Bistro does not (yet) provide a true compiler, the source-to-source translator generates an intermediate source file (ClassName.java). So, the overall syntactic structure of each class resembles that found in Java, including the location of classes with respect to their declared packages. The following patterns provide examples of Bistro class and interface (type) definition templates.

"Interface definition templates."
nil subtype: RootType metatype: [ "..." ] type: [ "..." ]
Supertype subtype: Subtype metatype: [ "..." ] type: [ "..." ]

"Class definition templates."
nil subclass: RootClass metaclass: [ "..." ] class: [ "..." ]
Superclass subclass: Subclass metaclass: [ "..." ] class: [ "..." ]

"Template for a class which implements an interface."
Superclass subclass: Subclass implements: Interface metaclass: [ "..." ] class: [ "..." ]

Notice that root classes and interfaces are derived from nil in Bistro. Root interfaces have no equivalent Java supertype, but root classes are derived from java.lang.Object and root metaclasses are derived from smalltalk.behavior.Class. Thus, the smalltalk.behavior.Object class is derived from java.lang.Object and its metaclass is derived from smalltalk.behavior.Class. See the discussion of metaclasses below for more details.

Support for object interfaces is one of the most powerful and innovative features of Java. Interfaces provide a language mechanism for defining polymorphism distinct and separate from inheritance. The object community recognized the importance of supporting object interfaces largely as a result of the development of distributed object computing facilities like CORBA and DCOM. Interfaces eliminate the need for multiple inheritance and the thorny issues associated with it. Because of their importance and significant value, Bistro also supports the definition and use of interfaces.


Packages

The absence of a name space mechanism is one of the major deficiencies of most commercial Smalltalk environments. The absence of name spaces permits the occurrence of class naming conflicts, especially when integrating large class libraries from independent software development organizations, including third party vendors. While some name space models have been proposed for Smalltalk14, 15, 16 none has yet been widely adopted by Smalltalk vendors.

Luckily, the Java language model supports the name space concept with packages. Packages are used to organize Java classes, but they also serve as name spaces for classes. This helps system designers resolve potential naming conflicts between classes. Java classes are logically organized and partitioned into packages. Java class files are physically organized into package directories in the file system. There is a direct correspondence between these logical and physical organizations. Figure 1 depicts these relationships.

packages contain classes

Figure 1. Logical vs. Physical Packaging

This model is simple and powerful. Bistro takes advantage of the Java package mechanism by duplicating the language mechanisms used to implement it. Java classes are compiled in the context of a package. The Java compiler uses the CLASSPATH environment variable to locate package directories. To support seamless integration with Java, Bistro also uses the CLASSPATH variable to locate package directories.

All classes defined in a package are immediately visible to each other. Public classes from other packages can be made visible by importing them. The Java import statement establishes visibility to individual classes or all the public classes contained in a package. Table 3 compares the packaging syntax of Bistro against that of Java.

Table 3. Packaging Comparison
Bistro Java
package: smalltalk.collection;
import: smalltalk.behavior.Object;
import: smalltalk.behavior.*;
import: smalltalk.magnitude.*;
package smalltalk.collection;
import smalltalk.behavior.Object;
import smalltalk.behavior.*;
import smalltalk.magnitude.*;

Consider a new class defined using the packaging information supplied in Table 3. The new class belongs to the Bistro package named smalltalk.collection. All uses of Object in the new class will refer to smalltalk.behavior.Object rather than java.lang.Object because of the fully qualified import statement. As with Java, imported classes may be declared as shown in Table 3, or their class names may be fully qualified by the names of the packages within which they were defined. So, instead of referring to the Character class imported by smalltalk.magnitude.*, it may also be fully qualified as smalltalk.magnitude.Character.


Implementing Metaclasses in Java

Smalltalk has first-class metaclasses, while Java does not. Bistro implements metaclasses by splitting each class definition in two parts - one Java class for the nominal class methods and data, and one Java class for metaclass methods and data. Putting Bistro metaclass methods and data in another Java class allows the metaclass hierarchy to parallel the nominal class hierarchy. This then supports inheritance and polymorphism for metaclasses like that found in Smalltalk.

Each Bistro metaclass is implemented as a public static inner class of a named Java class. The inner class name is the same as the outer class name with a dollar sign ($) prefix. Then, each class so defined has a public static member ($class) that refers to the sole instance of the metaclass. The following list shows the parallels between a few selected classes and metaclasses.

smalltalk.behavior.Object
smalltalk.collection.OrderedCollection
smalltalk.geometry.Point
smalltalk.behavior.$Object
smalltalk.collection.$OrderedCollection
smalltalk.geometry.$Point

Figure 2 shows the full inheritance hierarchy for some of the key classes from Smalltalk. Note each class has a static links to its metaclass ($class). However, these links must be resolved (by instantiation) after the inheritance links have been established (during compilation). This is made possible in the Java implementation by making each metaclass a public static inner class. Finally, the metaclass ($class) links of all metaclasses refer a singleton public static instance of the class Metaclass.

class and metaclass hierarchies

Figure 2. Classes and Metaclasses


Variable Declaration and the Scope of Names

Bistro mixes some of the language features from Smalltalk and Java. The design of Bistro attempts to achieve a balance between the features and limitations of both languages. Table 4 compares these features and limitations with respect to variable declarations.

Table 4. Variable Declarations
Smalltalk Bistro Java
declaration in outer scope allows use in inner scope yes yes yes
declare before use within a scope yes yes yes
declare local variables in a special section yes no no
variable declarations must be typed no no* yes
allows redeclaration (override) in inner scope no no* yes
allows initialization at declaration no yes yes

* Type information is optional in Bistro. Making it optional imposes the need to forbid variable redeclaration. To further simplify the language, Bistro eliminates the need to declare local variables in a separate section at the beginning of each block (as Smalltalk does). As with Java, variables can be declared anywhere in a scope, so long as they are declared before they are used, typically with an initial value.


Method Declarations

The syntax for Smalltalk methods and blocks are very close. The primary difference is that blocks are delimited with square brackets [ ], while methods are not. In order to support a declarative language model and normalize the syntax, Bistro uses square brackets for both. The declaration of Bistro classes, interfaces, and methods are all delimited with square brackets.

Normal Bistro method declarations are similar to those of Smalltalk. However, Bistro also supports primitive methods written with Java blocks. Both kinds of methods use signatures similar to those found in Smalltalk. The following code fragments illustrate these distinctions.

normalMethod: argument
[ "..."
]

primitiveMethod: argument
{ //...
}

Like Java, Bistro supports the use of several reserved words in the declaration of methods, including those that provide control over access, derivation, and thread synchronization. The following code fragments illustrate these features, which duplicate those found in Java.

private synchronized syncMethod: argument [ "..." ]

protected final finalMethod: argument [ "..." ]

native nativeMethod: argument []

abstract abstractMethod []

Notice that both abstract and native method declarations must have empty blocks because their implementations cannot be specified. Abstract methods must be implemented in subclasses. Final methods cannot be implemented (overridden) in subclasses. Native methods must be implemented using JNI.


Method Interoperability

To achieve seamless integration between Bistro and Java, their methods must be interoperable. So, Bistro code must be able to invoke methods in existing Java classes, and Java code must be able to invoke methods in Bistro classes. To achieve method interoperability, the compiler translates Bistro method names into Java method names using some renaming conventions. Special consideration must be given to both binary operators and keyword selectors.

The Java language does not (yet) support the use of operators for method names. However, there are some indications that such support may eventually be provided. The word operator has been reserved in the Java language specification. Also, while the language does not support them, Java class files and the Java VM both allow the use of operators for method names in the constant pool. So, once a full compiler that generates class files can be developed for Bistro, it will support the direct use of operators instead of renaming them (as currently implemented). Meanwhile, the standard binary operators supported by Smalltalk are renamed using the conventions shown in Table 5. Here again, the Bistro compiler adds a prefix ($) to the target method name to prevent confusion with existing Java protocols.

Smalltalk keyword selectors are also renamed using the conventions shown in Table 5. The colon in each keyword is replaced by an underscore, except for the final colon, which is dropped. Unlike Smalltalk, Bistro also supports the use of colons as argument separators, serving as anonymous keywords like those found in block signatures. These are dropped during translation, which allows Bistro to support the definition and invocation of Java methods that take more than one argument (see the ternary: example in Table 5).

Table 5. Method Renaming Conventions
Smalltalk Java
receiver unary
receiver binary: arg
receiver ternary: arg1 : arg2
receiver keyword: arg1 keyword: arg2
receiver.unary( )
receiver.binary( arg )
receiver.ternary( arg1, arg2 )
receiver.keyword_keyword( arg1, arg2 )
receiver + arg
receiver - arg
receiver * arg
receiver / arg
receiver // arg
receiver \ arg
receiver \\ arg
receiver @ arg
receiver , arg
receiver.$plus( arg )
receiver.$minus( arg )
receiver.$times( arg )
receiver.$div( arg )
receiver.$DIV( arg )
receiver.$rem( arg )
receiver.$REM( arg )
receiver.$at( arg )
receiver.$append( arg )
receiver = arg
receiver ~= arg
receiver == arg
receiver ~~ arg
receiver < arg
receiver <= arg
receiver > arg
receiver >= arg
receiver.$equal( arg )
receiver.$notEqual( arg )
receiver.$is( arg )
receiver.$isnt( arg )
receiver.$lessThan( arg )
receiver.$lessEqual( arg )
reciever.$moreThan( arg )
receiver.$moreEqual( arg )

Blocks

Blocks are a very powerful feature of the Smalltalk language. They are so flexible that they are used to implement all the Smalltalk control structures. This contributes to Smalltalk's syntactic simplicity. There are no reserved words for control structures as there are in other languages like Java and C++. Instead, the common control structures are expressed using standardized idioms with Blocks, Booleans, and Collections. Table 6 lists some of the most commonly used idioms as they are expressed in Bistro.

Table 6. Bistro Control Structures
Alternatives aBoolean ifTrue: [ "..." ]
aBoolean ifTrue: [ "..." ] ifFalse: [ "..." ]
aBoolean ifFalse: [ "..." ]
aBoolean ifFalse: [ "..." ] ifTrue: [ "..." ]
Loops [ booleanExpression ] whileTrue: [ "..." ]
[ booleanExpression ] whileFalse: [ "..." ]
[ "..." booleanExpression ] whileTrue
[ "..." booleanExpression ] whileFalse
Simple Iterators start to: end do: [ :index ! "..." ]
start to: end by: increment do: [ :index ! "..." ]
General Iterators aCollection do: [ :element ! "..." ]
aCollection select: [ :element ! "..." ]
aCollection reject: [ :element ! "..." ]
aCollection collect: [ :element ! "..." ]
Evaluations [ "..." ] value
[ :a ! "..." ] value: x
[ :a :b ! "..." ] value: x value: y

Notice that these standardized idioms do not include a switch or case statement. There are a number of ways to mimic such a control structure in Bistro (or Smalltalk). However, it is generally discouraged in favor of object-oriented designs that make use of classes and polymorphism to distinguish and implement separate cases.

Many of the control structures identified in Table 6 can be optimized during translation to Java. Often, they can be translated directly into equivalent Java control structures. Similar kinds of optimizations are often performed by existing commercial Smalltalk compilers. However, under certain circumstances, these control structures and other custom blocks are best implemented using Java inner classes.


Implementing Blocks with Inner Classes

Java inner classes make duplicating the semantics of blocks rather easy. Blocks can be implemented by translation into appropriate anonymous inner classes. By defining a few abstract base classes, Bistro provides the foundations for deriving these anonymous inner classes.

Some Smalltalk implementations are enriched in terms of the number of block arguments they support - i.e., they support an arbitrary number of block arguments. To keep the translation mapping simple, Bistro currently limits support for blocks to those that take 0, 1, or 2 arguments. Table 7 lists the abstract base classes for blocks included in Bistro and an example of the kind of block each supports.

Table 7. Bistro Block Base Classes
ZeroArgumentBlock [ "..." ] value
OneArgumentBlock [ :a ! "..." ] value: x
TwoArgumentBlock [ :a :b ! "..." ] value: x value: y

The following code fragment shows a portion of the Java class definition for the TwoArgumentBlock base class.

package smalltalk.behavior;
import smalltalk.behavior.Object;
public abstract class TwoArgumentBlock extends Object {
// ...
public abstract Object value_value(
final Object firstArgument, final Object secondArgument );
}

Given the foregoing class definition, the following example shows how a simple two argument block in Bistro is translated into a Java inner class.

"this Bistro method returns a commonly used block"
sortBlock
[
^[ :a :b ! a <= b ]
]
// the equivalent Java method looks like this
public Object sortBlock( ) {
return new TwoArgumentBlock() {
public Object value_value( final Object a, final Object b ) {
return a.perform_with( "$lessEqual", b );
}
}
}

Bistro supports optional typing, including method and block results and arguments. When translating typed block arguments to Java, the Bistro compiler uses type erasure by generating a generalized wrapper method in the inner class, as well as a typed method that contains the block code. The following example revisits the sortBlock method, this time with a typed block result and typed block arguments.

"this Bistro method includes a typed block result and arguments"
sortBlock
[
^[ !(Boolean) :a (String) :b (String) ! a <= b ]
]
// the equivalent Java method looks like this
public Object sortBlock( ) {
return new TwoArgumentBlock() {
// the wrapper method generalizes the argument types
public Object value_value( final Object a, final Object b ) {
// the wrapper invokes the typed method
return value_value( (String) a, (String) b );
}
// the typed method contains the block code
public Boolean value_value( final String a, final String b ) {
return a.$lessEqual( b );
}
}
}

Local Block Variables

Bistro local block variables can be conveniently mapped to instance variables in Java inner classes (when needed). The following example shows how a local block variable in Bistro is translated into an inner class instance variable in Java. The local block variable has been highlighted to help identify it.

"example of a Bistro local block variable"
extantElementsFrom: anArray
[
result := OrderedCollection new.
1 to: anArray size do:
[
:index !
"local block variable -->" element := anArray at: index.
element isNil ifFalse: [ result add: element ]
].
^result asArray
]
// the equivalent Java method looks like this
public Object extantElementsFrom( final Object aCollection ) {
final Object result = new OrderedCollection();
primitive.IntegerFrom( 1 ).to_do(
anArray.perform( "size" ),
new OneArgumentBlock() {
Object element; // <-- local block variable!
public Object value( final Object index ) {
element = anArray.perform_with( "at", index );

if( !primitive.booleanFrom(
element.perform_with( "isNil" )
)) {
return result.perform_with( "add", element );
}
return primitive.$nil();
}
}
);
return result.perform( "asArray" );
}

Methods Returns from Blocks

Smalltalk supports the ability to return method results directly from inside nested blocks using the caret (^). Smalltalk method returns exit all enclosing block scopes, including the enclosing method scope. The following search: method provides an example of this capability.

search: aCollection for: searchTargets
[
aCollection do:
[
:element !
( searchTargets includes: element )
ifTrue: [ ^element ]
].
^nil
]

The generalized iterator do: in the foregoing method cannot be optimized because the message receiver, aCollection, must implement it. When the Bistro compiler cannot optimize the blocks inside a method and uses inner classes to implement them, special consideration must be given to the method returns. Bistro implements method returns using the Java exception handling facility. Bistro provides a special subclass of the java.lang.RuntimeException class that carries a method result across the scopes of the intervening inner classes. Once it reaches the enclosing method scope, the smalltalk.behavior.MethodResult is caught and unwrapped to return the method result. So, the Bistro compiler translates the search: method into the following Java code.

public Object search_for( final Object aCollection, final Object searchTargets ) {
try {
aCollection.perform_with( "do",
new OneArgumentBlock() {
public Object value( final Object element ) {
if( primitive.booleanFrom(
searchTargets.perform_with( "includes", element )
)) {
return element;
}
}
}
);
return primitive.$nil();
} catch( MethodResult r ) {
return r.returnValue;
}
}

Notice that the method arguments must be declared final in order to be used inside the value method of the inner class. In order to make this opportunity for argument usage available to inner class blocks, the Bistro compiler makes all method and block arguments final during translation.

The foregoing sample also shows one clear advantage Smalltalk has over Java. It clearly takes more Java code to express the equivalent method semantics, even after permissible block optimizations are performed during translation, and especially when dynamic typing is involved.


Dynamic Typing with Reflection

The Java reflection facility provides information about object types at runtime. Bistro uses the Java reflection facility to determine the type of an object and to resolve method invocations at run time (late binding). The foregoing code fragment for the sortBlock method provides an example of how the Bistro compiler translates dynamically typed messages. The Bistro compiler takes advantage of the standard Smalltalk perform: idioms to implement dynamically typed message sends. As in Smalltalk, an unimplemented method discovered at runtime ultimately produces a doesNotUnderstand: message.

Of course, the most significant issue associated with dynamic typing is performance. Experimental measurements indicate that raw reflective method invocations are about 100 times slower than normal method invocations. This is significant, but not necessarily prohibitive. A number of possible improvements offer themselves.

All of these contributions make the exploration of dynamic typing for the Java VM worthwhile. Also, given that Sun has made the Java VM available through a community source license, it may even be possible to customize the VM to accelerate method lookup for dynamically typed languages like Bistro. Ultimately, Bistro allows performance critical methods to utilize type information to improve speed, or they can be hand coded as primitive Java methods.


Exceptions

While some superficial similarities exist between Smalltalk and Java exception handling mechanisms, there are also some fundamental differences. So, deciding whether and how to integrate these mechanisms presents one of the more challenging problems for the design of Bistro. Table 8 compares the salient aspects of the Smalltalk and Java exception handling mechanisms. Table 9 considers the Smalltalk exception handling idioms in detail.

Table 8. Smalltalk and Java Exceptions
Smalltalk Java
models exceptions as instances of exception classes (Signal and its subclasses) models exceptions as instances of exception classes (Throwable and its subclasses)
has no special language elements for dealing with exceptions has language elements for dealing with exceptions:
try { } catch { } finally { }, throw and throws
provides fine grained control over exception handling, including the ability to decide whether and when stack frames are unwound, and the ability to resume execution at the point of origin (where the exceptional Signal was raised) exception handling is strictly stack oriented - once unwound, the stack frames between the exception origin (throw) and the handler (catch) are unavailable
exceptions never impact method signatures the classes of all exceptions thrown within a method must be declared in a throws clause in the method signature, except for those exceptions whose classes are derived from java.lang.RuntimeException
Table 9. Smalltalk Exception Idioms
[ "aBlock" ] ensure: [ "finalBlock" ]. Evaluates aBlock and guarantees that the finalBlock will be executed, even if an exception throws control out of the first block. This idiom is useful for ensuring that resources are released properly even under exceptional circumstances.
[ "aBlock" ] ifCurtailed:
[ "exceptionHandler" ]
Evaluates aBlock. If an exception throws control out of the first block, the exceptionHandler will be evaluated. This idiom is useful for catching all exceptions.
[ "aBlock" ] on: ExceptionClass
do: [ :exception | "exceptionHandler" ]
Evaluates aBlock. If an exception of type ExceptionClass throws control out of the first block, the exceptionHandler will be evaluated with the exception instance as its argument.

Finding the common ground between Smalltalk and Java is easy, but merging the mechanisms completely may not be feasible. Still, some of the Smalltalk idioms can easily be mapped to Java mechanisms. For example, the ZeroArgumentBlock class can provide the following primitive methods.

ensure: terminationBlock (ZeroArgumentBlock)
{
Object result = null;
try {
result = this.value();
} finally {
terminationBlock.value();
}
return result;
}
ifCurtailed: exceptionBlock (ZeroArgumentBlock)
{
Object result = null;
try {
result = this.value();
} catch( smalltalk.behavior.Signal s ) {
exceptionBlock.value();
}
return result;
}

To support the exception handling capabilities of Java, a new control structure idiom has been developed for Bistro. This new idiom mirrors the structure of the Java exception handling syntax, to which the compiler translates uses of the new idiom. Note how the catch: phrase extends the existing ensure: idiom.

[ "aBlock" ]
catch: [ :exception (ExceptionClass) !
"exceptionHandler"
]
"... additional catch phrases ..."
ensure: [ "finalBlock" ].

The compiler translates this kind of message directly into Java code using the following pattern.

try { aBlock.value(); }
catch( ExceptionClass exception )
{ exceptionHandler.value( exception ); }
//... additional catch phrases ...
finally { finalBlock.value(); }

Synchronized Objects

Java supports thread synchronization on methods and within methods using the synchronized reserved word. Bistro also supports both kinds of thread synchronization. As noted in the language overview, Bistro supports the declaration of synchronized methods. But, Bistro also supports object synchronization within methods. The base class, smalltalk.behavior.Object, provides the following primitive instance method.

acquireMonitorDuring: aBlock (ZeroArgumentBlock)
{
synchronized( this ) {
return aBlock.value();
}
}

Thus, any Bistro method may synchronize threads on an object by using a statement similar to the following one.

anObject acquireMonitorDuring: [ "..." ].

Current Project Status and Future Plans

Bistro currently consists of a pair of source-to-source translators developed in Smalltalk and Java, as well as a library of classes.

The Bistro grammar is stable and feature complete, although it will likely need revisions if true byte code compiler is developed. The grammar will need the addition of language elements to support primitive Java methods for a true compiler.

The Bistro class library includes many of the behaviors needed to support Smalltalk execution in a Java VM, including the classes and metaclasses shown in Figure 2.

The primary project goals going forward are:

  1. Complete conversion of enough Smalltalk classes to support real development, including the essential behavior classes, magnitudes, collections, and streams. Ultimately, this should include all the behaviors specified in the ANSI Smalltalk standard - NCITS 319:199x.4

  2. It may be desirable to develop a full Bistro compiler that supports the direct translation of Bistro code into Java class files - i.e., code that will run directly in a Java VM. However, maintaining the Bistro compiler as a source-to-source translator also has advantages. It's simpler and easier to integrate with existing commercial Java compilers and development tools.

  3. The Smalltalk translator needs further refinement. Several issues need to be addressed. Primary among these are how to support the assignment of Smalltalk classes to packages and potential improvements in generated code that may be provided by type annotations and type inference.

  4. Develop an integrated programming and debugging environment for Bistro. This should duplicate as much as possible the level of integration found in Smalltalk - i.e., recompilation of individual methods without noticeable delays and immediate expression evaluation. These features combine to produce the most rapid code / test cycles available in the industry.

Conclusion

So, is Smalltalk still relevant? It appears to have been marginalized, but only in comparison with Java. The Smalltalk market has continued to grow along with (though not nearly as quickly as) the Java market. Java and its widely available VM clearly offer opportunities for the further evolution of Smalltalk. Bistro offers Smalltalk the opportunity of riding the further growth and pervasion of the Java market.


Acknowledgments

Thanks to Tracy Tondro for all his help and sanity checks with some of the thornier language design choices. Thanks to Jean-François Cloutier, Dan Rubel, Steve Ginsburg, Allen Wirfs-Brock, and Ward Cunningham for their interest and encouragement.


Bibliography

  1. Jeff Sutherland. The Smalltalk Manifesto: Avoiding RoadKill on the InfoBahn. June, 1996. <http://www.jeffsutherland.org/papers/smallman.html>
  2. Alan Wirfs-Brock, Brian Wilkerson. An Overview of Modular Smalltalk. OOPSLA Conference Proceedings. ACM, September 1988.
  3. Allen Wirfs-Brock, Juanita Ewing, Harold Williams and Brian Wilkerson. A Declarative Model for Defining Smalltalk Programs. OOPSLA conference talk given October, 1996. <http://www.bell-labs.com/people/cope/oopsla/#AllenWirfsBrock>
  4. X3J20 workgroup. ANSI Smalltalk. NCITS 319:199x. December, 1997. http://www.stic.org/STLANG/standard_v1_9.PDF
  5. Per Bothner. Translating Smalltalk to Java. November, 1996. <http://home.pacbell.net/bothner/smalltalk.html>
  6. Mark Fussell. Java and Smalltalk Syntax Compared. 1997. <http://www.chimu.com/publications/JavaSmalltalkSyntax.html>
  7. Steve McClure. Smalltalk Strengths Stand Out. International Data Corporation white paper for the Smalltalk Industry Council (STIC), 1997. <http://www.stic.org/Research/IDC97/stic.htm>
  8. Nik Boyd. Toward Smalltalk and Java Language Integration. May, 1997. <http://home.labridge.com/~nikboyd/papers/sttojava/>
  9. Andrew Eidsness. Generating Java Class Files from Smalltalk Objects. 1997. <http://www.cyberus.ca/~eidsness/jasper/>
  10. Claus Gittinger, Stefan Vogel. Smalltalk/Java-Integration in Smalltalk/X. 1997. <http://www.exept.de/stja97_en.html>
  11. Georg Elsner, Reinhard Behrendt. Smalltalk to Java Converter. January, 1998. <http://www.orisa.de/s2jengl.html>
  12. Tom Krauß. S2J: A Tool-Set to Migrate Smalltalk Applications to Java. June, 1998. <http://www.gebit.de/s2j/s2j.htm>
  13. Nik Boyd. Introducing Jist. February, 1999. <http://home.labridge.com/~nikboyd/papers/jist/>
  14. Nik Boyd. Modules: Encapsulating Behavior in Smalltalk. The Smalltalk Report 2(5). SIGS Publications, February, 1993.
  15. Wayne Beaton. Name Space in Smalltalk/V for Win32. The Smalltalk Report 4(1). SIGS Publications, September, 1994.
  16. Nik Boyd. Class Naming and Privacy in Smalltalk The Smalltalk Report 6(3). SIGS Publications, November, 1996.

Trademarks

Java is a trademark of Sun Microsystems, Inc.
Visual Smalltalk is a trademark of ObjectShare, Inc.
Smalltalk/X is a trademark of Exept Software AG.