This tutorial shows how easy it is to define an external DSL .
The tutorial is based on the Xtext framework as it is contained in
oAW 4.2. Make sure that you have installed oAW 4.2 into an Eclipse 3.3
workbench running on Java >=5. The easiest way to get a working IDE is
to download the Eclipse 3.3 distribution for RCP/Plug-in developers from
eclipse.org and the oAW all-in-one release from oAW's
website.
The purpose of this tutorial is to illustrate definition of external DSLs using Xtext. The process we're going to go through will start by defining an Xtext grammar for our DSL, generate a parser, a meta model and a corresponding text editor out of it. Afterwards we will see how to refine the DSL and it's editor using provided facilities. In the end we will see how one can generate code out of textual models.
The actual content of this example is rather trivial – we will generate Java classes following the Java Beans conventions. The model will contain entities (such as Person or Address ) including some attributes and relationships among them – a rather typical data model. From these entities in the model we want to generate the Beans for implementation in Java. In a real setting, we might also want to generate persistence mappings, etc. We will not do this for this simple introduction.
We'll define a DSL for simple domain models following some concepts of Domain-Driven Design.
We don't want to define too much semantic for each of these concepts here because this tutorial is about how to develope a DSL not why. In addition, most of the semantics of a DSL's concept is usually implemented in a generator or an interpreter, which is not the focus of this turorial.
We use this example because the concepts are well known (even if the meaning is not formally defined). So we don't need to explain them or motivate the example but can focus on other things.
The following UML diagram shows the abstract syntax of the DSL we're going to define:
To get started we need to create some projects. Therefore we use the provided Xtext wizard:
The wizard dialog lets you specify a number of properties used for different things. See the Xtext reference documentation for detailed information about what the different properties mean. For the moment just ensure that everything looks like shown in the screenshot above (i.e. stay with the defaults) and click "Finish".
The wizard generates three projects. The first one
(my.dsl) is the language project. Therein we'll
define the grammar and Xtext's generator will place the meta model and
the parser in it.
The second one is named my.dsl.editor and
contains the Eclipse text editor for our DSL. For now it is empty
because, we haven't defined a DSL and started the generator so
far.
The third project (my.dsl-generator) is called
the generator project. Actually this is not directly related to Xtext
but to oAW in general. That means that the Xtext generator doesn't
generate a single file into this project. However, the wizard has placed
a workflow file using the DSL parser (which we will generate in a
second) as well as a default Xpand template. So we don't need to do any
configuration manually.
Your navigator should look similar to the one in this screenshot:
The wizard created and automatically opened an Xtext grammar file
(mydsl.xtxt). Therein we have to specify the grammar
for our textual DSL. Using Xtext's grammar language one defines the
abstract syntax (i.e. the meta model) and the conrete syntax of the the
language. This allows for very short turn-arounds when refactoring and /
or improving the DSL.
The root element of all expressions made in our new DSL is
called a Model. The is the name of the parser rule
which is invoked for each textual model to be parsed.
The parser rule Model creates and returns an
instance of the corresponding meta model element (also called
'Model'). A Model just consists of
a list of types (rule Type). In
the model the list of types can be accessed through the reference
types. When it comes to code generation one can
evaluate expressions such as 'myModel.types' to get
the types of a model.
A Type corresponds to an abstract meta type,
because it just refers to two other rules (without assigning the result
to a property like Model does). So the list of
Types consists of Datatypes
and/or Entities.
A Datatype starts with keyword
"datatype" followed by an identifier
(ID). The value of the identifier is assigned to the
Datatype's property name.
ID is a built-in rule and is similar to a Java
identifier (i.e. a word starting with a letter followed by
alphanumerical characters and/or underscores).
An Entity starts with keyword
"entity" followed by an identifier
(ID) which is assigned to the property
name as well. In addition an entity contains
declarations of owned features. The list of declarations is assigned to
the reference called 'features' and it is surrounded
by curly brackets ("{" and
"}").
Last but not least a Feature consists of an
Identifier (ID) which refers to a
Type and another identifier which specifies the name
of the Feature.
Make sure to read the reference documentation in order understand how a metamodel is derived from a grammar and how the linking for cross references actually works.
Now that we have defined an Xtext grammar for our simple DSL we
can start Xtext's generator by right clicking the workflow file
(generate.oaw). It will create the meta model and the
parser as well as fill the editor project with the needed
artifacts.
We now can start a so called runtime workbench in order to see our newly defined DSL and it's editor in action. To do so click the "Debug..."-Action in the toolbar like it is shown in the next screenshot. Select the entry called "Open Debug Dialog...".
When the dialog opens select "Eclipse Application" on the left and create a new configuration by clicking the "new" icon (the upper left icon). Leave everything as it's initially configured and press "Debug".
The generated editor already provides a lot of default funtionality which has only been derived from the grammar.
Some of them are code completion, "Got To Declaration", "Find References", outline view, error marker support and folding as shown in the following screenshots:
The most important thing every DSL designer should do in addition to defining the grammar is to specify so called checks. A check (also called invariant) is expressed in oAW's Check language and is a declarative constraint for all model elements of a certain type. There are some built-in checks Xtext derives from the grammar such as verifying that a referenced element (in our example the reference from Feature.type to Type) could found. We can't and don't want to add all the checks to the grammar because it would get too complex. Instead the Xtext generator has created an empty check file for us where we are supposed to add our semantic constraints.
Let's ensure that each Type has a unique name and within an Entity each Feature has a unique name as well:
The first thing you usually do in a check file is to import the
used meta models (import mydsl). The next line
(extension org::example::dsl::Extensions) imports an
extension file. Extension files contain so called extensions, which are
essentially functions.
The syntax of a check starts with the keyword
"context" followed by the meta type (e.g.
Type) we want to apply the constraint to. Then one
has to specify whether the severity is ERROR or WARNING using the
appropriate keyword. A message for the user follows. Note that you can
use the model element which is actually checked through the imlied
'this' variable. This means we could have written "Duplicate
type "+this.name instead.
The condition which should hold (i.e. evaluate to true) for all instances of the given type is separated by a colon. The expressions used here (and in the message) are oAW expressions. See the reference documentation of the core languages (Check is one of them) for details.
We use two extensions here. The first one is called
allElements() and has been generated by Xtext. It
returns a list of all model elements containd in the model.
typeSelect(Type) selects all elements which are of
type Type. From that list we select all elements which have the same
name as the actually checked model element. The constraints ensure that
the size of the reulted list is equal to 1 (i.e. the resulted list only
containes the currently checked model element).
The second extension is used in the check for features. Basically
the condition is very similar to the one we discussed before. The only
difference is that we don't want to check whether the name is unique
within the whole model but within all features of the current feature's
entity. To access the feature's entity we have defined an extension
'entity()' in the file
org/example/dsl/Extension.ext like so:
Note that we reexport all extenions from
GenExtensions here. That's why we can refer to
'allElements()' from within the check file. We can
now start the runtime workbench again without starting Xtext's generator
once more because we didn't change the grammar file. Try to break one of
the constraints:
The constraints are not only checked within the editor but are checked when invoking the parser from oAW's workflow engine as well.
There are more things we could do to enhance the editor (e.g. outline view, code completion or navigation). Have a look at the reference documentation to see how this works.
Now that we have a DSL we may want to do something useful with it. DSLs are essentially small programming languages. A programming language has to be understandable by a computer. There are basically two ways to make a language "understandable" by a computer. The first one is to write a compiler which transforms expressions made in one language into another language which is already understandable by a computer. For example a Java compiler transforms Java programs to ByteCode programms. ByteCode is understandable because there are VMs which translate expressions in Java ByteCode into more native instructions. This is usually done at runtime. Translating a language at runtime is called interpretation (ignoring special cases like Just-in-Time compilation here).
With Xtext models one can either create a compiler (a.k.a. generator) or an interpreter. Although there are good reasons for both approaches we will just discuss how one creates a generator in this tutorial.
The Xtext wizard already created a generator project for us. We're
going to write an Xpand template which generates simple JavaBeans from
our entities. It's assumed that there is a Java datatype corresponding
to the datatypes used in the models (e.g. String). So
we don't need to care about mapping data types.
So just open the Xpand template (Main.xpt) and
and modifiy it like this:
The definition main is invoked from the workflow file. It's
declared for elements of type mydsl::Model, which
corresponds to the root node of our DSL models. Within this definition
another definition (javaBean) is called
(<<EXPAND javaBean...) for each model element
(...FOREACH...) contained in
Model's reference 'types' which is
of type Entity
(typeSelect(Entity)).
The definition javaBean is declared for elements of type Entity. Therein we open a file (<<FILE ...). The path/name of the file is defined through an expression. In this case it corresponds to the name of the entity suffixed with '.java'. It's going to be generated into the src-gen directory directly.
All text contained between <<FILE ...>> and <<ENDFILE>> will go to the new file. Xpand provides control statements (FOR, IF, ELSEIF,...) as well as evaluation of expression in order to create the desired code. See the core languages reference documentation for details.
Now we've defined a DSL and a corresponding Generator. We don't need to use it in Eclipse - you can run the generator from ant, maven, command line, etc. as well - we'll now show you how to install everything into Eclipse in order to start a new project based on your DSL and generator.
Right click on any project in the navigator view. Choose 'export'.
Choose "Deployable plug-ins and fragments".
Choose the directory where you've installed Eclipse, mark all three projects and click the "Finish"-button.
When the export process has finished restart your workbench. Now choose a different workspace. Within the new workspace you can create a new 'mydsl' project using the generated wizard.
The wizard creates an Eclipse plug-in project containing the needed dependencies to the dsl bundle, the generator bundle and the oAW bundles. It also generates a workflow which invokes the workflow from our generator as well as an empty model file. Let's fill the model file with some information.
Now we can run the workflow by right clicking it. Choose
Run As->oAW Workflow.
The generator finished successfully and the src-gen folder contains the generated Java classes.
That's it for now. Feel free to play around with this small example. If you like it I'ld propose you have a look at the reference documentation and try to create your own DSL.
Have fun!