How do ccpn data models get written

Here are my brief results from digging around the ccpn data-model production software (v2.1) from source forge to see how data-model writing gets carried out (note these are my random doodlings and may not represents how it really works i.e. ymmv)

1. you call python makeJavaApi.py from the command line which lives in ccpn/python/memops/scripts_v2
   
2. This then uses XmlModelIo.readModel() to read the pure ccpn metamodel which returns the root meta-package as its result
   
3. a thing called the ModelPortal gets constructed from the meta-model. (This seems to provide functions that access the meta-model in particular ways, examples include for example leafPackagesByImport - leaf packages sorted by import (imported before importing), dataTypesByInheritance - data types sorted by inheritance (supertype before subtype), dataObjTypesAlphabetic - data types sorted alphabetically by name etc...)
   
4. JavaFileModelAdapt.processModel(modelPortal) processes the model-portal (and its owned meta-model) to adapt it to the idiosyncrasies of the requirements of the java language
   
5. the class method JavaFileApiGen.writeApi gets called with the model-portal as its first parameter and information in the other parameters about where to store the model the version.
6. finally inside JavaFileApiGen.writeApi we do some more setup create a JavaFileApiGen object and then call processModel() on it this then calls a general meta-model traverser which is in the class ModelTraverse.processModel
7. now comes the interesting bit! This then causes a series of callbacks on the JavaFileApiGen class to be called. Using the following hack I think I can work out most of them...
   grep def ../metamodel/ModelTraverse_py_2_1.py | tr '(' ' ' | awk '{print $2}'
   
8. this then visits the following list of apparently interesting methods
   
   • processBranchPackage
   • initLeafPackage
   • processLeafPackage
   • endLeafPackage
   • processDataType
   • processConstant
   • processException
   • initClass
   • processClass
   • endClass
   • initDataObjType
   • processDataObjType
   • endDataObjType
   • processAttribute
   • processRole
   • initOperation
   • processOperation
   • endOperation
   • processParameter
   which seem to correspon well with the parts of the datamodel... ;-)
   

Don't invite eclipse euorpa to the feast!

Eclipse is a wonderful thing and the update manager can make life easy. However, if you want to install something a little out the way and unsupported you can so easily get in a mess under Europa (I guess this is why P2 is such a big thing for Ganymede)

Why Europa isn't my quartermaster!

1. If you have the wrong platform bundles installed ie x86 rather than x86_64 specifically
   
   • org.eclipse.core.filesystem.linux.x86_64_1.0.100.v20070510.jar
   • org.eclipse.equinox.launcher.gtk.linux.x86_64_1.0.2.R331_v20071019
   • org.eclipse.platform.source.linux.gtk.x86_64_3.3.2.R33x_v20071022-_19UEksF-G8Yc6bUv3Dz
   • org.eclipse.rcp.source.linux.gtk.x86_64_3.3.2.R33x_r20071022-8y8eE9CEV3FspP8HJrY1M2dS
   • org.eclipse.swt.gtk.linux.x86_64_3.3.2.v3347.jar
     
     the error message you get on startup is very obtuse
     
     
   
   exit code code=13 is about as friendly as my favourite error message 'an unknown error occurred at an unknown location'
   
   
2. I can't see where it says i have the wrong platform modules here... isn't this something the launcher should check for?

Then things can get worse ;-)

I tried to swap the x86 bundles for the X86_64 versions I had lying around and then got this...

now if you understand what you have just done this is OK. However, it's still not the most useful message! What should a launchers companion library be called (Yhe Charlotte Bartlett.dll)

So how did you get me in this mess Stanley?

Now the way I got into this mess wasn't so obvious either....

I decided I wanted to play with emf and use the Soyatec eUML2 free editor. This all seems well and fine so I did what they asked and added the Soyatec website to my update site list and tried to install the eUML2 component. Thats when the hell started as I tried to get the right set of dependencies for the soyatec product to work with the other plugins on the Europa Discovery Site and failed over and over on dependencies on gmf and ocl so for example I would get

eUML2 (*.*.*.*) requires feature ***.***.gmf (x.x..xxxx) or compatible.

where x.x..xxxx was a lower version number than the one I already hadinstalled....

More heartache

So having got into dependany hell where did I go next. I remembered that Yoxos/Inoopract has this damn neat Yoxos on demand service and stuff me it does what it says on the jar:

• it sorted my dependencies like a treat
• it made me my own custom eclipse which I can go back to
• it was free
• Yoxos has its own provisioning which is available inside eclipse and works again like a dream. It resolves dependencies and just making it all work (in actual fact this plug-in appears to be very much related to the online experience which appears to be a RAP version of the Yoxos provisioning plug-in)
  

so can you see the problem (remembering the problems I discussed above), here is a screen shot of the yoxo online provisioning web app that might give you a clue:

Yep I am Linux_x86_64 whereas the button here is effectively Linux_86. I guess this is something Innoopract/Yoxos need to solve or at least they do need to flag that they don't support x86_64 linux.

Wrapup

So what can we conclude from this trip round the houses?

• getting the right versions of eclipse plug-ins can still be a hassle even with the update manager and the newest dependency management tools
• If you have a system which doesn't exactly match the rest of the world (though i might say come on x86_64 isn't that rare ;-) you can easily end up in trouble
• the best laid plans of mice and men fail especially when you plan to 'just quickly' install a piece of software (alway a bad move)

Random thoughts

• I suceeded in the end by doing the off-line Soyatec installation which included all the required dependencies at the right version. Heres the picture to prove it:
  
  
  
  
  

• Why is the update manager in the help menu??? It always seemed strange to me; File maybe Window maybe but help no way!!
• Why doesn't the eclipse Updates window have a maximize button some of the package names are long enough....
  
  
  
  
  

  (and yes I know I can expand it it just isn't convenient)
  
  
• why don't I blog more often (no don't ask me to answer that one)
  

installing scipy and numpy with analysis

scipy and numpy provide sophisticated and fast mathematical routines for python and installing them with analysis should be a breeze. However, it wasn't quite as simple as it seemed ;-) Specifically analysis is built with narrow unicode support and scipy/numpy require wide support....

so you have to add --enable-unicode=ucs4 to the python configure command line when compiling python for analysis using the supplied python. This command this is at about line 346 in installCode.py fro analysis 1.0.10

cmds = []
cmds.append('./configure --enable-unicode=ucs4 --prefix=%s/%s' % (top_dir,python_rel_dir))
cmds.append('make')
cmds.append('make install')
runCmds(cmds)

in the function def compilePython(x11_abs_dir, tcl_abs_dir, tk_abs_dir):

what to do if an update fails

ccp analysis has a very nice update feature. However, if it fails you can get in a state where you can't start analysis to get the updates. So here are two simple cures (one is from Tim and uses a python interface, the second is just how to get to the update site with a web browser to pull down raw files).

To run the update agent without analysis running

1. initialise you analysis environment so the ccpn root directory is set e.g. setenv
   

   CCPN_HOME ... in csh syntax or export CCPN_HOME='...' in sh syntax

2. if need be setup you pythonpath to include the ccpn directories
   

3. python $CCPN_HOME/python/ccpnmr/update/UpdatePopup.py and proceed as usual

to use a web browser...

1. http://www.bio.cam.ac.uk/~ccpn/ccpNmrUpdate
2. this directory will contain files of the form python__temp_ccpnmr__temp_analysis_.py where would be Util for Util.py for example
3. download save and install
   

analysis on dapper drake & Edgy Eft

Installing analysis on ubuntu dapper drake (ubuntu 6.06LTS) is not a clean affair for various murky reasons (none of which are to do with the ccpn chaps by the way just life and code rot).

So here goes

Prerequisites

1. you need gcc and g++ installed (packages gcc and g++-4.0)
   
2. you need glut installed (packages freeglut3, freeglut3-dev)
3. you need the x development headers and extension installed (packages libext6 and libxext-dev libx11-dev; more on this later)
4. you must have build-essential installed if you want to compile your own python
   

Installation

1. don't try to use the installation of tcl/tk that comee from ubuntu they don't play well with the ccpn installation script (tcl/tk lib and include are not both under the same directory)
2. don't use the tcl/tk 8.3 files that come with ccpnmr 1.0.10 tas hey have configure scripts that are broken under the latest versions of bash
3. get tcl8.4.13-src.tar.gz and tk8.4.13-src.tar.gz http://www.tcl.tk/software/tcltk/8.4.html
4. mkdir tk8.4 and tcl8.4 in ccpnmr1.0
5. place thetcl/tk tar files in their respective directories
6. gunzip them
7. remove the -src part of the name e.g. tk8.4.13-src.tar.gz -> tk8.4.13.tar.gz

now edit the installation script installCode.py

1. line 27 tcltk_version = '8.4' #was tcltk_version = '8.3'
2. line 28 tcltk_release = '8.4.13' #was tcltk_release = '8.3...'

fiddle with the X extension libraries

1. /usr/lib/libXext.so doesn't exists
2. so become root
3. ln -s libXext.so.6 libXext.so

setup the environment variables changing tcl /tk 8.3 to 8.4

1. setenv LD_LIBRARY_PATH\ ${CCPNMR_TOP_DIR}/tcl8.4/lib:${CCPNMR_TOP_DIR}/tk8.4/lib
2. setenv TCL_LIBRARY {$CCPNMR_TOP_DIR}/tcl8.4/lib/tcl8.4
3. setenv TK_LIBRARY {$CCPNMR_TOP_DIR}/tk8.4/lib/tk8.4

carry out your normal installation using installCode.py

all done and here is the proof (and no I didn't fake it in photoshop ;-))

notes

1. all paths entered have to be absolute
2. the X11 directory is /usr

Minimise menu bar problem

Under KDE you can minimise the Analysis window. However, it doesn't have an entry in KDE's list of windows (only spectra get this), so you can't maximise it again. Workround top.deiconify() in the shell...

Chains and LinkSequentialSpinSystemsPopup

LinkSequentialSpinSystemsPopup ddidn't report chains for residues if there is more than one chain in the MolSystem. It does now ;-)



However, I have only modified it so chains will show when there are multiple chains present...

A feature with backup

If you have auto backup on in analysis and the directory you backup to is missing. If you try to switch auto backup off it will ask you if you want to create the directory... if you then say no it will say the directory can't be created and so it is not setting up automatic backups... If you say yes and you are on the wrong computer of course it will fail to create the directory and throw an error.

conclusion it seems to be doing input validation at the wrong places and assuming all exits require validation

Strange ones

A couple of strange corners in the ccpn datamodel

1. Project doesn't have a parent Object field
2. there are isotopes and isotopecodes
1. isotopes are objects
2. isotope codes are fixed length strings of the form 1H 13C etc
3. you can't get an isotopecode from an isotope and vice versa without coding...
   

munging readline in analysis

some programs such as IPython and faetures such as command line hcacking require gnu readline. Here is what I did under ubuntu to get it going

1. all this is using the version of python that you use to run analysis (you may need to set the environment variables TCL_LIBRARY etc to get i to work (see $CCPNMR_TOP_DIR/bin/analysis)
2. this is using ccpns own downloaded python
3. change to $CCPNMR_TOP_DIR/python2.4/Python-2.4.2/Modules/
4. edit Setup and uncomment line at about line 160 which reads 'readline readline.c -lreadline -ltermcap'
5. cd to $CCPNMR_TOP_DIR/python2.4/Python-2.4.2
6. run python setup.py install

Get ipython in analysis

1. download ipython http://ipython.scipy.org/dist/
2. extract to your ccpnm directory tar ipython-0.7.1.fix1.tar.gz
3. go to tar ipython-0.7.1
4. type setup.py install where is the python you used to install analysis
5. edit $CCPNMR_TOP_DIR/ccpnmr1.0/python/ccpnmr/analysis/AnalysisGui.py
6. at about line 52 after import Tkinter put import IPython on a new line
7. at about line 105 add:
   from IPython.Shell import IPShellEmbed
   ipshell = IPShellEmbed()
   ipshell()
   

analysis tables widget

Two problems here

1. the tables widget creates new rows on the fly (I guess) so the widths of the columns are constantly jumping around
2. if you hide the rows in a table widget you get nice little + signs but which column is which when you want to reexpand them. Wouldn't a popup be better in general....

analysis spin systems editor

analysis, edit spin system dialog: you can display strips and display cells... So why is it missing a button with a name something like 'display resonances'.

Creation

Let there be light