You are here: Home V2 Software Software More ... Memops Code Generation Documentation 2007 Modeling HowTo

2007 Modeling HowTo

Detailed tutorial for modeling and code generation with ObjectDomain and Memops framework. Dates from 2007. OUT OF DATE, but still the best available for a general introduction.

Modeling with Memops

Contents:

Getting the CCPN code:

Installing ObjectDomain:


Set-up:


Using ObjectDomain:


Modeling Introduction:

Classes
Attributes
Links
Multiplicity
Changeable
Names
Keys
Inheritance

Advanced Modeling:
Getting started
DataTypes
Package import and interpackage links
ContentStored packages
Constraints
Operations
Special Constructor and Destructor code
Derived attributes and links
Automatic attributes and links
Special attributes
One-way links
Links that are their own inverse

Code Generation:

Saving the model to disk
Generating code

Getting the CCPN code:

For those who want to do modeling, the CCPN repository including the Memops generation machinery, is on SourceForge at http://sourceforge.net/projects/ccpn/. Download it in the normal way. I shall refer to the topmost directory as 'cvsroot'; it should contain directories doc/, python/, model/, etc.

If you do not want to model, but only to use the generated APIs, we recommend you download from http://www.ccpn.ac.uk/downloads/downloads.html - the installation script there will make it easier for you.

Installing ObjectDomain :

ObjectDomain is a nice UML editor, but the company has gone out of business. CCPN holds a couple of unused ObjectDomain licenses, so you can get a copy from us legally. Upgrading to a new UML editor is among our future plans.

Set-up:

You need Python 2.6 or higher to run the CCPN programs.

For set-up you need to have the system python directory and the top level python directory of the CCPN code tree on your PYTHONPATH.

When you have installed ObjectDomain, go into the ObjectDomain directories, and edit the file toplevel/python/scripts/odUserSetup.py,
adding the equivalent of the following two lines - you have to change it to fit with your own file locations:

import sys
sys.path.append('/home/rhf22/CCPN/cvsroot/python')

using the full path to cvsroot/python.

Using ObjectDomain:

The screen is divided in five areas:

·         The diagram view (top right), where most of the editing gets done.

·         The Python shell, (bottom right), for running scripts and anything you want a Python shell for.

·         The contents tree (top left), to select where you want to go.

·         The current contents, (middle left), showing the contents of what is curently selected in the contents tree. Things with two ends, such as links, inheritance, etc. will be shown as contained in one (and only one) of the classes they connect.

·         The world view (bottom left) to move and zoom the diagram view.

Controls and places to look for things are generally intuitive.

·         Double clicking an object in the diagram view, contents tree, or world view will open the edit object popup, where you set most attributes of objects. To edit the appearance in the diagram of an object, use right mouse in the diagram view, and select 'Edit View'.

·         Double clicking a diagram in the content tree or current content will open it.

·         Some actions, like creating a new class diagram, can best be done from the contents tree, e,.g. right mouse->create->Class Diagram on the containing package.

·         Moving an existing class into a diagram is done by dragging it from the contents tree.

Deleting an object from a diagram view will only remove the image - when you remove the last image you will be asked whether to remove the object from the model. Deleting an object form the contents tree will delete it permanently.

Standard modeling actions are done from the menu bar below the diagram view. The one you will need most are 'New Class' (second from left), 'Create Link' (unadorned line with two right angles), 'Make Inheritance' (fully drawn broadheaded arrow pointing up), and 'Make Parent/Child Link' ('Create Link' with filled black diamond). The only other one you will ever need is 'Create Dependency' (dotted arrow pointing up and right).

When using the 'edit object' popup, text that is typed is not stored unless you click on something else in the popup before you close. A bit of experimenting will get you used to this.

A specific problem is that the normal way of removing a superclass from inside the 'edit object' popup does not work. You can do it by using right mouse 'delete' in the current contents area, or by selecting and deleting the inheritance arrow on the diagram view.

Modeling Introduction:

The description here is simplified. More details are given in 'Advanced Modeling' below. Note that the generation scripts (see below) will check that your model is valid, and given mostly sensible error messages to tell you what is wrong.

Classes:

The first thing you will need is to create a class. A valid class needs a number of things done.

·         All classes must have a superclass (set in superclass tab inside popup). The superclass should be Implementation.NormalStored by default, or another class that inherits from it.

·         All non-abstract classes must have a parent link. The parent must be either another class within the same package or Project. Make a link with a filled diamond on the parent. The link must be 'Aggregation:Composite, Multiplicity:1, Changeable:frozen' on the parent side, and 'Multiplicity:*' on the child side.

·         All Attributes, links, and operations in a class share the same namespace and so must have different names. This includes inherited attributes etc.

Attributes:

These are added in the popup of the class. All attributes must have a name and a Data Type. DataTypes are set by clicking the three dots button to the right of TypeRef. 'Normal' classes can not be used as Data Types for Attributes. At the test level you should stick to the data types defined in the model.

Links:

Links are made by connecting two classes on the diagram. The two ends of a link are edited separately (set 'Assoc Ends' tab after double clicking on a link). Names need not be set, as they are set from the names of the connected classes by default. It is sometimes confusing which end of a link means what. An example should clarify this. If we look at the link between Implementation.Url and Implementation.AbstractStorage, the end connected to Url has the number 1. This means that each AbstractAStorage has exactly one Url. The other end, marked '*' and 'storages' means that each Url can have any number of storages, and that the name used for the link is 'storages', not 'abstractStorages'.

Multiplicity:

This denotes the maximum and minumum number of values allowed for an attribute or link. If the multiplicity is not set, it defaults to '0..1' (optional). A single number (e.g. 1 or 5) means exactly that many (1..1 or 5..5). '*' means 'as many as you like'. The most common multiplicities are '0..1', '1', and '*', but you could have any continous range (e.g. '3..17').

Changeable:

If Changeable is set to Frozen, this means the attribute or link must be set when the object is created and cannot be modified later.

Names:

Names of functions etc. use singular or plural as appropriate. For attributes or links that may have more than one value, the name given should be plural. To give the singular form, the attibute or link should have a Tagged Value, with the tag being 'baseName' and the value being the singular form of the name. For links this is only necessary if you give an explicit name, as the names are otherwise derived from the class name. You need not bother about these tagged values if you stick to single values for attributes and avoid explicit names for links.

Keys:

Each non-abstract class must have one or more attributes or links specified as keys. These must uniquely identify the object among other objects of the same type with the same parent. The key is specified by adding the tagged value 'mainkey' to the class. The value of the tagged value is a comma-separated list of the names of the attributes and links that make up the key. All elements of the key must be frozen and have multiplicity n..n (e.g. 1..1, 2..2). If you have no good candidate for the key, you should use the serial number. This must be an attribute called 'serial', DataType 'int', multiplicity '1', changeable 'frozen. It must furthermore have a tagged Value 'isAutomatic':'True'.

Inheritance:

The contents of a class is copied down to the subclass, so they share a namespace. You can only have name overlap (overwriting) in special cases. The key and parent link are also inherited and cannot be overwritten.

Advanced Modeling:

When you are modeling for real you need to consider a few extra things. This is still an abbreviated version, though. See the header of MetaModel.py for details, or use the existing model as an example.

Getting started:

You should use the full model, Model.odm for real work.

You could delete some of the existing packages, but we recommend that you simply work in a new one - if you ever want to link your work to the existing stuff you will have to anyway.

You either make a new package in Logical), or decide to work inside 'ccp' (other packages are reserved for the implementation or for other groups). You now make a new package for your work inside the package you have selected. The package must have a Tagged Value 'shortName':'XXXX”, where XXXX is an upper case name of no more than four characters that is not already in use. This name serves as a package identifier when making e.g. database names.

Now you create a class diagram inside your package (right mouse->create->class diagram). You rename the class diagram '_<myPackage>-details' , to fit with the diagram naming convention. You can have several diagrams, but they should all start with underscore and end with -details. When your work is stable, you can then make overview diagrams without the '-details' as has been done for the rest of the model. It will be easier to put a copy of Project in your diagram, so you can make parent links to Project. You can get the small version of the class by using the options under right-mouse->Presentation Options

DataTypes:

·         To create a dataType, create a class, then set Stereotype to DataType (class popup, class tab, left side).
DataTypes do not obey the same rules as Classes.

·         All Data Types must be subclasses of another DataType, inheriting eventuallly from one of the basic types (currently Int, Long, Float, Double, Boolean, String). You can use DateTime even though it is not implemented yet. It will be at some point, but meanwhile it is a subclass of String. At present Long and Double are not used, and are implemented like Int and Float, respectively, This may change.

·         DataTypes can be further specified by tagged values. In addition to Constraints (see below) you can set the length (string types only). This is the maximum allowed length of the string. Also you can set enumeration and isOpen, the latter requires the former. This serves to make enumerated data types. The enumeration tagged value is a comma-separeated tuple of strings (with quotation marks) that gives the allowed values. Note that an enumerated set of integers would be given as e.g. ('1','3','5'), the fact that they are integers follows from the fact that the datatype would inherit from Int. If isOpen is True the enumeration is Open. This means that you can use values not in the enumeration, so that it is more like a recommended series of values.

Package import and interpackage links:

If you make a link between classes in two different packages, you must tell which package imports from which. The only exception is Implementation, which is imported by default. The direction of import matters - basically if A imports B this means that package A knows about package B, and that importing A will trigger the import of B, whether we are talking about API code or data. B, on the other hand, knows nothing about A. Clearly new packages should import old packages, not the other way around. To set an import, look at the three diagrams in ccp: _Citation, _LIMS, and _Structural. Put your package on a diagram next to the package it should import. Draw a dependency arrow (dotted arrow pointing top right on menu) from importing to imported package. Double-click the arrow and set stereotype to 'import'. As a graphics convention right click the arrow, do
Presentation Options->Stereotype Label: Off, and do Diagram->Refresh Diagrams from the top menu.

Constraints:

You can constrain your data beyond what e.g. types and multiplicities can do. This is done by entering code snippets that are slotted into the generated API code in an appropriate context. Depending on the situation, the variables 'self' and 'value' will be defined in context and can be used. Constraints must either be single lines of code that evaluate to a truth value, or multiple lines of code that set the parameter isValid to a truth value. There are three kinds of Constraints

·         Constraints on Data Types: These can depend only on the 'value' parameter

·         Constraints on Attributes and Links. These can depend both on 'value' and 'self'. As they are evaluated during object creation, you can not, however, count on other values in the object being set. The only value that can be relied on is the link to the parent, that is always set first.

·         Constraints on a class. These are evaluated only at the end of object creation or when checkValid is called explicitly, and so can rely on the object being in a legal state.

Constraints on datatypes, attributes and links are evaluated before any operation that modifies the appropriate value, and are thus guaranteed to hold. Constraints on classes are evaluated only at the end of object creation and file loading, or when the checkValid function is called explicitly. Therefore these constraints may be temporarily broken.

See the header comment to MetaModel for the precise rules on how to enter Constraints.

Operations:

Most operations are generated automatically. After all, the code needed to set or get a text attribute with a given name can be deduced from the model description. But, if you want, you can enter custom versions of autogenerated functions or your own new operations in the model.

·         Operations are entered in the class that carries them (operation tab).

·         To overwrite an autogenerated operation just put in one with the same name.

·         If the operation has no side effects (e.g. get, findFirst, findAll), set 'Query' to True.

·         All operations must have an opType tagged value. Operations that correspond to autogenerated ones can have the opTypes 'get' or 'getByKey' or 'getBuNavigation', 'set', 'add', 'remove', 'findFirst', 'findAll'. Other operations must have opType 'other'. You cannot overwrite creation, deletion, or factory functions, but you can add extra code to them. The 'getByKey' and 'getByNavigation' opTypes are alternative ways of specificating 'get' functions. he code entered inthese cases will be handled differently by the generation machinery. Please sese existing exampls for details.

·         Operation parameters are not set for functions that override autogenerated ones – only for completely new functions. A single return parameter must be called 'result', and must be of the type (class or datatype) of the value returned. The return type is the same whether a single value or a list is returned. A single input parameter must be called 'value' - the rules for typing are the same for input and return parameters.

·         Operations must have two tagged values code:python and code:java that hold the code to be executed.

See the header comment to MetaModel.py for the detailed rules for operations.

Special Constructor and Destructor code:

It is possible to enter code that is added to constructors and destructors.

·         For constructors 'hasSpecialConstructor':True signals that there is this kind of code. constructorCode:python and constructorCode:java tags contain the actual code. Special constructor code is executed after construction is finished (so the object is otherwise valid) but before validation checking. This code is not executed when isReading is set to True. A typical use of this facility would be to automatically create extra required objects when several objects must be created together.

·         For destructor code the relevant tags are hasSpecialDestructor, destructorCode:python and destructorCode:java. This code is executed at the beginning of the delete functions before any deletion has taken place. A typical use would be code that prevented deletion of objects as long as certain links are set.

Derived attributes and links:

It is possible to enter attributes and link elements that are never stored but calculated on the fly.

·         They are entered in the normal way, but attributes are given the tagged value 'isDerived':True. For links the tagged value 'isDerived':True is given to the *association*, not to either of the Association Ends.

·         For a derived element there must be an Operation that explicitly overrides the 'get' function for the element.

·         Normally the derived element is set to 'frozen'. If this is not done there must be an Operation that explicitly overrides the 'set' function for the element. This requires, obviously, that the setter function so modifies the underlying stored parameters that you would afterwards get the value you had just set. An example might be e.g. a point position that could be given as either x,y or r,theta. Here one of the coordinate sets (say, Cartesian) could be stored, while the attributes for the other one (Polar) could be derived but settable.

·         If a derivation does not return a value the behavior depends on the multiplicity. For a mandatory element you get an error - for an optional one you return None (or an empty tuple).

Automatic attributes and links:

The tagged value 'isAutomatic':True signifies an attribute that is stored in the normal way, but that cannot be set because it is set automatically (and discreetly) by the API, by constructorCode etc.. The main example is the 'serial' attribute

Special attributes:

·         serial: serves to provide a key for classes that otherwise do not have one. It is described above how to define serials. Serials are set automatically when a new object is created, and a new value should not be passed in. A new serial will be larger than the serial of any similar child of the same parent. Serials of deleted objects are not reused, except in rare cases if you delete the object with the highest serial, save the file and quit. Documentation for serials is entered automatically.

·         details: documentation is entered automatically. The details attribute should be of type Text or String. By convention it is used for free text comments, notes, etc.

One-way links:

The 'navigable' checkbox in the edit popup for links (Assoc ends) determines if a link is navigable in a given direction. In the Model description there will be a MetaRole only if the link is navigable. In one-way links the object on the other end does not know there is a link to it. There is no way to follow the link in the inverse direction, and if the linked-to object is deleted the link is left dangling. Unless derived, one-way links should be used only for links to reference data that do not change(e.g. chemical element descriptions) or where the nature of the data gives some guarantee that the linked-to object will not be deleted.

Links that are their own inverse:

It is possible in special circumstances to make a link from a class to itself where there is only one role. Examples would be covalently-bound-to (for atoms) of currently-married-to (for people) where the link and the inverse link are the same. See MetaModel.py for details.

Code Generation:

Saving the model to disk:

When you think the model is ready, you should save it to disk. Note that the procedure does not remove previous model files, so if you remove or rename packages you must remove the old model files and all related autogenerated code by hand. You go into the Python Shell in ObjectDomain and type the following:

from memops.scripts_v2.model import ObjectDomain
from memops.scripts_v2.model import XmlModelIo
mm = ObjectDomain.modelFromOd()
XmlModelIo.writeModel(mm)

The first two lines need only be done the first time. Once you get to the fourth line things are OK and you can quit ObjectDomain and go on to the next step.

The third line is where the model Python objects are generated and the validity checking is done.
If an error happens at this stage, you have made a mistake in entering the model and should go and fix it. This always happens a few times. The validity checking is very thorough. If you get a 'style warning' you have not followed the normal guidelines for model object names. We recommend that you fix these too.

Sometimes you will want to exclude some of the packages from your code generation. There are draft packages in the model (such as 'ccp.Crystallography') or you may want to do without packages for areas that you do not need (such as NMR). This can be a little tricky; packages import each other, and if you include package x, you automatically get all packages that package x imports, recursively. Here you have to know your way around model to be sure about what you are getting. But if you want to avoid some packages (and are sure the imports are OK, you are responsible) you can use the command (e.g.):
PyModelGen.writeModel(mm, excludePackageNames=('ccp.Crystallography'))

Generating code:

The python/memops/scripts_v2/makePython.py file is a make script that will generate all python-related files. Go to whatever directory you like, and type

python <path-to-file>/makePython.py

If you get any errors, there is presumably an error in your model, which you should try to fix, but this should happen only rarely.

You will get a lot of warning messages:

WARNING, <modelelement> <full name> has no documentation
is given for every model element with no documentation. This is to nag you to document your model

WARNING: File '/home/rhf22/CCPN/cvsroot/model/doc//html/diagram/ccp_Crystallography__Crystallography_Diag.gif' does not exist
tells you either that you do not have all the diagrams with the correct name (pairs named _XXX and _XXX-details) for the diagram-xml generator, or that you have not yet written the diagrams from ObjectDomain to the correct location. Ignore it - you only need these things done when you want to use the machinery for making releases.

The last thing the script does is to import all python files in cvsroot/python and its subdirectories, in order to check for errors (note that you should avoid files that actually do things when imported, because of this). For each import error, you will get a message like e.g.

WARNING, Import failed for ccpnmr.analysis.AnalysisPopup

You should check what is actually wrong by typing

python ccpnmr/analysis/AnalysisPopup.py

This will show you the actual error message. If there is an error in any file that has to do with your model, you most likely have an error in one of the code snippets you added (constraints or methods). This should be fixed. If the errors only concern code that you do not recognise there may be no cause for alarm. You can have errors at this stage for a number of reasons: A file may depend on a c-coded module that you have not compiled.; There may be checked-in scripts that only work in a particular environment, like personalised make scripts for the developers or like the scripts that create the original ChemComp reference data; A modified file may not work pending the check-in of another modified file (this is a live repository, it changes all the time); There may be an error in someone else’s code. If you find any errors that do not have an obvious explanation please let us know.

Any warnings about files in ccp.gui, ccp.format, ccpnmr.format, memops.editor, ccp.util.Molecule, memops.scripts.JavaApiGen, memops.scripts.JavaXmlGen, memops.scripts.SqlSchemaGen, memops.scripts.makeModel as well as WARNING directory /home/rhf22/CCPN/testroot/python/ccp/model lacks file '__init__.py' or the same warning for ccpnmr/model.

Data Model Versions:

There is a version number for the entire data model. It is defined in the variable memops.general.Constants.currentModelVersion. This version is updated by hand. It should be updated before every new release if the model has changed, and every time the data backwards compatibility code is changed. In internal development it may be left unchanged even though the model changes.