Modeling with Memops

Contents:

Getting the CCPN code:

Installing ObjectDomain:

Set-up:

Using ObjectDomain:

Test Modeling:
    Classes
    Attributes
    Links
    Multiplicity
    Changeable
    Names
    Keys
    Inheritance

Advanced Modeling:
    Getting started
    DataTypes
    Package import and interpackage links
    ContentStored packages
    Constraints
    Operations
    Special Constructor and Destructor code
    Derived attributes and links
    Automatic attributes and links
    Special attributes
    One-way links
    Links that are their own inverse

Code Generation:
    Saving the model to disk
    Generating code

Getting the CCPN code:

For those who want to do modeling, the CCPN repository including the Memops generation machinery, is on SourceForge at http://sourceforge.net/projects/ccpn/. Download it in the normal way. I shall refer to the topmost directory as 'cvsroot'; it should contain directories doc/, python/, model/, etc.

If you do not want to model, but only to use the generated APIs, we recommend you download from http://www.ccpn.ac.uk/downloads/downloads.html - the installation script there will make it easier for you.

Installing ObjectDomain :

You can get hold of ObjectDomain through http://www.objectdomain.com. There is a 30-say test version, but this does not allow you to save if you have more than 30 classes in your model, which is barely enough for testing. For any serious work you need a paid-for version.

Set-up:

You need Python 2.2 or higher to run the CCPN programs.

For set-up you need to have cvsroot/python in your PYTHONPATH. In the tcsh shell I have :
setenv PYTHONPATH /home/rhf22/CCPN/LIB/ccpnmr/python2.2/Python-2.2.3:/home/rhf22/CCPN/cvsroot/python

When you have installed ObjectDomain, go into the ObjectDomain directories, and edit the file toplevel/python/scripts/odUserSetup.py,
adding the following two lines:

import sys
sys.path.append('/home/rhf22/CCPN/cvsroot/python')

using the full path to cvsroot/python.

Using ObjectDomain:

The screen is divided in five areas:

Controls and places to look for things are generally intuitive.

Deleting an object from a diagram view will only remove the image - when you remove the last image you will be asked whether to remove the object from the model. Deleting an object form the contents tree will delete it permanently.

Standard modeling actions are done from the menu bar below the diagram view. The one you will need most are 'New Class' (second from left), 'Create Link' (unadorned line with two right angles), 'Make Inheritance' (fully drawn broadheaded arrow pointing up), and 'Make Parent/Child Link' ('Create Link' with filled black diamond). The only other one you will ever need is 'Create Dependency' (dotted arrow pointing up and right).

When using the 'edit object' popup, text that is typed is not stored unless you click on something else in the popup before you close. A bit of experimenting will get you used to this.

A specific problem is that the normal way of removing a superclass from inside the 'edit object' popup does not work. You can do it by using right mouse 'delete' in the current contents area, or by selecting and deleting the inheritance arrow on the diagram view.

Test Modeling:

For a quick test with the trial version of ObjectDomain, go into ObjectDomain and open cvsroot/model/od/memops/Test.odm. Save it as something else before making changes.

Now open 'Logical' in the contents tree, and go into the _Test-details diagram in the testing.Test package. This is where you should do your modeling. You will also have the memops.Implementation package available but you should make no chages there.

The description here is simplified. More details are given in 'Advanced Modeling' below. Note that the generation scripts (see below) will check that your model is valid, and given mostly sensible error messages to tell you what is wrong.

When you get to generating code from your model, you should note that Java generation does not work with the minimum test model. Also, you need to remove all files cvsroot/python/*/model/*.py, as the model files you download from SurceForge are ntoe compartible with the minimum test model.

Classes:

The first thing you will need is to create a class. A valid class needs a number of things done.

Attributes:

These are added in the popup of the class. All attributes must have a name and a Data Type. DataTypes are set by clicking the three dots button to the right of TypeRef. 'Normal' classes can not be used as Data Types for Attributes. At the test level you should stick to the data types defined in the model.

Links:

Links are made by connecting two classes on the diagram. The two ends of a link are edited separately (set 'Assoc Ends' tab after double clicking on a link). Names need not be set, as they are set from the names of the connected classes by default. It is sometimes confusing which end of a link means what. An example should clarify this. If we look at the link between Implementation.Url and Implementation.AbstractStorage, the end connected to Url has the number 1. This means that each AbstractAStorage has exactly one Url. The other end, marked '*' and 'storages' means that each Url can have any number of storages, and that the name used for the link is 'storages', not 'abstractStorages'.

Multiplicity:

This denotes the maximum and minumum number of values allowed for an attribute or link. If the multiplicity is not set, it defaults to '0..1' (optional). A single number (e.g. 1 or 5) means exactly that many (1..1 or 5..5). '*' means 'as many as you like'. The most common multiplicities are '0..1', '1', and '*', but you could have any continous range (e.g. '3..17').

Changeable:

If Changeable is set to Frozen, this means the attribute or link must be set when the object is created and cannot be modified later.

Names:

Names of functions etc. use singular or plural as appropriate. For attributes or links that may have more than one value, the name given should be plural. To give the singulsar form, the attibute or link should have a Tagged Value, with the tag being 'baseName' and the value being the singular form of the name. For links this is only necessary if you give an explicitl name, as the names are otherwise derived from the class name. You need not bother about these tagged values if you stick to single values for attributes and avoid explicit names for links.

Keys:

Each non-abstract class must have one or more attributes or links specified as keys. These must uniquely identify the object among other objects of the same type with the same parent. The key is specified by adding the tagged value 'mainkey' to the class. The value of the tagged value is a comma-separated list of the names of the attributes and links that make up the key. All elements of the key must be frozen and have multiplicity n..n (e.g. 1..1, 2..2). If you have no good candidate for the key, you should use the serial number. This must be an attribute called 'serial', DataType 'int', multiplicity '1', changeable 'frozen. It must furthermore have a tagged Value 'isAutomatic':'True'.

Inheritance:

The contents of a class is copied down to the subclass, so they share a namespace. You can only have name overlap (overwriting) in special cases. The key and parent link are also inherited and cannot be overwritten.

Advanced Modeling:

When you are modeling for real you need to consider a few extra things. This is still an abbreviated version, though. See the header of MetaModel.py for details, or use the existing model as an example.

Getting started:

You should use the full model, Model.odm for real work, as Test.odm was a minimum version designed to fit under the 30 class limit of the ObjectDomain test version.

You could delete some of the existing packages, but we recomemnd that you simply work in a new one - if you ever want to link your work to the existing stuff you will have to anyway.

You either make a new package in Logical (similar to 'testing' in Test.odm), or decide to work inside 'ccp' (other packages are erserved for the implementation or for other groups). You now make a new package for your work inside the package you have selected. The package must have a Tagged Value 'shortName':'XXXX”, where XXXX is an upper case name of no more than four characters that is not already in use. This name serves as a package identifier when making e.g. database names.

Now you create a class diagram inside your package (right mouse->create->class diagram). You rename the clas diagram '_<myPackage>-details' , to fit with the diagram naming convention. You can have several diagrams, but they should all start with underscore and end with -details. When your work is stable, you can then make overview diagrams without the '-details' as has been done for the rest of the model. It will be easier to put a copy of Project in your diagram, so you can make parent links to Project. You can get the small version of the class by using the options under right-mouse->Presentation Options

DataTypes:

Package import and interpackage links:

If you make a link between classes in two diferent packages, you must tell which package imports from which. The only exception is Implementation, which is imported by default. The direction of import matters - basically if A imports B this means that package A knows about package B, and that importing A will trigger the import of B, whether we are talking about API code or data. B, on the other hand, knows nothing about A. Clearly new packages should import old packages, not the other way around. To set an import, look at the three diagrams in ccp: _Citation, _LIMS, and _Structural. Put your package on a diagram next to the package it should imoprt. Draw a dependency arrow (dotted arrow pointing top right on menu) from importing to imported package. Double-click the arrow and set stereotype to 'import'. As a graphics convention right click the arrow, do
Presentation Options->Stereotype Label: Off, and do Diagram->Refresh Diagrams from the top menu.

ContentStored packages:

Most packages are stored as a single XML file, but some are stored in several files, like ChemComp and Coordinates. These are more complicated. A ContentStored packge must have a HeadObject. This must be a class that is a subclass of Implementation.ContentStored, and that is a child of Project. The HeadObject is not part of the ContentStored package. All Classes in the ContentStored package must be children of either the HeadObject or an other class in the ContentStored package. It does not matter for the data loading whether the class with the HeadObject imports the ContentStored package or the other way around; it will make a difference for the loading of code.

Constraints:

You can constrain your data beyond what e.g. types and multiplicities can do. This is done by entering code snippets that are slotted into the generated API code in an appropriate context. Depending on the situation, the variables 'self' and 'value' will be defined in context and can be used. Constraints must either be single lines of code that evaluate to a truth value, or multiple lines of code that set the parameter isValid to a truth value. There are three kinds of Constraints

Constraints on datatypes, attributes and links are evaluated before any operation that modifies the appropriate value, and are thus guaranteed to hold. Constraints on classes are evaluated only at the end of object creation and file loading, or when the checkValid function is called explicitly. Therefore these constraints may be temporarily broken.

See the header comment to MetaModel for the precise rules on how to enter Constraints.

Operations:

Most operations are generated automatically. After all, the code needed to set or get a text attribute with a given name can be deduced from the model description. But, if you want, you can enter custom versions of autogenerated functions or your own new operations in the model.

Special Constructor and Destructor code:

It is possible to enter code that is added to constructors and destructors.

Derived attributes and links:

It is possible to enter attributes and link elements that are never stored but calculated on the fly.

Automatic attributes and links:

The tagged value 'isAutomatic':True signifies an attribute that is stored in the normal way, but that cannot be set because it is set automatically (and discreetly) by the API, by specialConstructorCode etc.. The main example is the 'serial' attribute

Special attributes:

One-way links:

The 'navigable' checkbox in the edit popup for links (Assoc ends) determines if a link is navigable in a given direction. In the Model description there will be a MetaRole only if the link is navigable. In one-way links the object on the other end does not know there is a link to it. There is no way to follow the link in the inverse direction, and if the linked-to object is deleted the link is left dangling. Unless derived, one-way links should be used only for links to reference data that do not change(e.g. chemical element descriptions) or where the nature of the data gives some guarantee that the linked-to object will not be deleted.

Links that are their own inverse:

It is possible in special circumstances to make a link from a class to itself where there is only one role. Examples would be covalently-bound-to (for atoms) of married-to (for people) where the link and the inverse link are the same. See MetaModel.py for details.

Code Generation:

Saving the model to disk:

When you think the model is ready, you should save it to disk. Note that the procedure does not remove previous model files, so if you remove or rename packages you must remove the old model files and all related autogenerated code by hand. You go into the Python Shell in ObjectDomain and type the following:

from memops.scripts import ObjectDomain
from memops.scripts import PyModelIO
mm = ObjectDomain.modelFromOd()
PyModelIO.writePythonCode(mm)

The first two lines need only be done the first time. Once you get to the fourth line things are OK and you can quit ObjectDomain and go on to the next step.

The third line is where the model Python objects are generated and the validity checking is done.
If an error happens at this stage, you have made a mistake in entering the model and should go and fix it. This always happens a few times. The validity checking is very thorough. If you get a 'style warning' you have not followed the normal guidelines for model object names. We recommend that you fix these too.

Generating code:

The python/memops/scripts/makePython.py file is a make script that will generate all python-related files. Go to whatever directory you like, and type

python <path-to-file>/makePython.py

If you get any errors, there is presumably an error in your model, which you should try to fix, but this should happen only rarely.

You will get a lot of warning messages:

WARNING, <modelelement> <full name> has no documentation
is given for every model element with no documentation. This is to nag you to document your model

WARNING: File '/home/rhf22/CCPN/cvsroot/model/doc//html/diagram/ccp_Crystallography__Crystallography_Diag.gif' does not exist
tells you either that you do not have all the diagrams with the correct name (pairs named _XXX and _XXX-details) for the diagram-xml generator, or that you have not yet written the diagrams from ObjectDomain to the correct location. Ignore it - you only need these things done when you want to use the machinery for making releases.

The last thing the script does is to import all python files in cvsroot/python and its subdirectories, in order to check for errors (note that you should avoid files that actually do things when imported, because of this). For each import error, you will get a message like e.g.

WARNING, Import failed for ccpnmr.analysis.AnalysisPopup

You should check what is actually wrong by typing

python ccpnmr/analysis/AnalysisPopup.py

This will show you the actual error message. If there is an error in any file that has to do with your model, you most likely have an error in one of the code snippets you added (constraints or methods). This should be fixed. If the errors only concern code that you do not recognise there may be no cause for alarm. You can have errors at this stage for a number of reasons: A file may depend on a c-coded module that you have not compiled.; There may be checked-in scripts that only work in a particular environment, like personalised make scripts for the developers or like the scripts that create the original ChemComp reference data; A modified file may not work pending the check-in of another modified file (this is a live repository, it changes all the time); There may be an error in someone elses code. If you find any errors that do not have an obvious explanation please let us know.

If you are using the minimum test model (Test.odm), remember you need to remove all the python/*/model/*.py files before you start generating the model. Also, you will get a lot of warnings. A lot of the existing code in the repository will not work with the minimum model, and it would be dangerous to change the code just to adapt to a quick0-test code. In this situation you should ignore:

Any warnings about files in ccp.gui, ccp.format, ccpnmr.format, memops.editor, ccp.util.Molecule, memops.scripts.JavaApiGen, memops.scripts.JavaXmlGen, memops.scripts.SqlSchemaGen, memops.scripts.makeModel as well as WARNING directory /home/rhf22/CCPN/testroot/python/ccp/model lacks file '__init__.py' or the same warning for ccpnmr/model.

Data Model Versions:

There is a version number for the entire data model. It is defined in the variable memops.general.Constants.currentModelVersion. This version is updated by hand. It should be updated before every new release if the model has changed, and every time the data backwards compatibility code is changed. In internal development it may be left unchanged even though the model changes.