- IMP-users - salilab.org

Re: [IMP-users] installing within python version 2.6
by Ben Webb 11 Oct '10

11 Oct '10

On 10/8/10 4:17 PM, jeremy adler wrote: > Thanks for your help. I tried building it from the source into the > /usr/lib/python2.6 directory (after deleting everything from my previous > attempt). The follwoing files are placed there:\ > > _IMP_algebra.so _IMP_display.so _IMP_helper.so _IMP_saxs.so > _IMP_atom.so _IMP_domino.so _IMP_misc.so _IMP.so > _IMP_benchmark.so _IMP_em.so _IMP_modeller.so _IMP_statistics.so > _IMP_container.so _IMP_example.so _IMP_multifit.so > _IMP_core.so _IMP_gsl.so _IMP_restrainer.so > > Though nothing else is. When i try to import imp now it says the module > isnt there. Any advice? These are just the interfaces between Python and the underlying IMP C++ modules. There should also be a bunch of "real" Python files (.py extension) in the IMP subdirectory. These should also have been installed by "scons install" at the same time as those .so files above. Maybe they went in a different directory for some reason? Check the output from "scons install". Ben -- ben(a)salilab.org http://salilab.org/~ben/ "It is a capital mistake to theorize before one has data." - Sir Arthur Conan Doyle

1 0

installing within python version 2.6
by jeremy adler 08 Oct '10

08 Oct '10

I have tried installing the version available for linux-it goes directly into the python v2.4 folder so I copied it to the v2.6 folder. I get the following error when i try to access IMP import IMP Traceback (most recent call last): File "<stdin>", line 1, in <module> File "/usr/lib/python2.6/IMP/__init__.py", line 25, in <module> _IMP = swig_import_helper() File "/usr/lib/python2.6/IMP/__init__.py", line 21, in swig_import_helper _mod = imp.load_module('_IMP', fp, pathname, description) ImportError: libimp.so: cannot open shared object file: No such file or directory The _IMP.so is present. Is there something else im missing? thanks Jeremy

2 1

IMP svn compiling issues Mac OS X 10.6 fink-64
by Benjamin SCHWARZ 27 Sep '10

27 Sep '10

Hi list I try a compilation of IMP svn 6742 on Macbook laptop OS X 10.6 * Fink 64 bits installed (in /sw) with packages : cgal-4 fftw-3 boost1.35.python26 boost1.35.nopython gsl and graphviz * Ann hand installed in /usr/local * Modeller installed as most recent MAC dmg And I have several problems : 1.) the compilation process fails on cgal polyhedron compilation g++ -o modules/cgal/src/internal/polyhedrons.os -c -Os -fno-common -dynamic -I/BenInstalls/imp-install/imp-debug/include -I/opt/local/include -I/sw/include -Wall -Wno-deprecated -Woverloaded-virtual -g -fno-finite-math-only -ftrapping-math -fsignaling-nans -fno-float-store -frounding-math -fno-strict-aliasing -DCGAL_DISABLE_ROUNDING_MATH_CHECK -fvisibility=hidden -fPIC -DIMPCGAL_EXPORTS -DGCC_VISIBILITY -Ibuild/include -I/BenInstalls/imp/src/build/include -I/usr/local/include -I/sw/include -I/usr/local/ann/ann_1.1.2/include /BenInstalls/imp/src/modules/cgal/src/internal/polyhedrons.cpp In file included from /sw/include/CGAL/Nef_3/SNC_items.h:26, from /sw/include/CGAL/Nef_3/Default_items.h:21, from /sw/include/CGAL/Nef_polyhedron_3.h:29, from /BenInstalls/imp/src/modules/cgal/src/internal/polyhedrons.cpp:8: /sw/include/CGAL/Nef_3/Halfedge.h: In member function 'typename Refs::Vector_3 CGAL::Halfedge_base<Refs>::vector() const': /sw/include/CGAL/Nef_3/Halfedge.h:110: error: 'ORIGIN' is not a member of 'CGAL' /usr/include/c++/4.2.1/bits/stl_function.h: In member function 'bool std::less<_Tp>::operator()(const _Tp&, const _Tp&) const [..] CGAL awful template insult [..] /BenInstalls/imp/src/modules/cgal/src/internal/polyhedrons.cpp:100: instantiated from here /usr/include/c++/4.2.1/bits/stl_function.h:227: error: no match for 'operator<' in '__x < __y' [..] a few lines with replacement propositions for this operator [..] Enabled dependencies: python, ANN, NetCDF, FFTW3, GSL, CGAL, Boost.ProgramOptions Disabled/unfound optional dependencies: Boost.FileSystem, ExampleLib, Bullet Enabled modules: kernel, cgal, algebra, core, container, atom, display, domino, em, example, misc, saxs, helper, statistics, gsl, modeller, restrainer, domino2, benchmark Disabled modules: em2d, multifit, bullet Errors building: modules/cgal/src/internal/polyhedrons.os 2.) I was then able to continue the compilation by disabling the cgal dependency (cgal="no" in the config.py). It then goes on quite a long time then crashes at link time on : g++ -o applications/saxs/profile.o -c -Os -fno-common -dynamic -I/BenInstalls/imp-install/imp-debug/include -I/opt/local/include -I/sw/include -Wall -Wno-deprecated -Woverloaded-virtual -g -Ibuild/include -I/BenInstalls/imp/src/build/include -I/usr/local/include -I/sw/include -I/usr/local/ann/ann_1.1.2/include /BenInstalls/imp/src/applications/saxs/profile.cpp g++ -o applications/saxs/foxs -g applications/saxs/profile.o applications/saxs/Gnuplot.o -Lbuild/lib -L/BenInstalls/imp/src/build/lib -L/usr/local/lib -L/sw/lib -L/usr/local/ann/ann_1.1.2/lib -L/BenInstalls/imp-install/imp-debug/lib -limp_saxs -limp_atom -limp_container -limp_core -limp_algebra -limp -lANN -lnetcdf_c++ -lboost_program_options ld: warning: directory '/BenInstalls/imp/src/build/lib' following -L not found ld: warning: directory '/BenInstalls/imp-install/imp-debug/lib' following -L not found Undefined symbols: "boost::program_options::options_description::options_description(std::basic_string<char, std::char_traits<char>, std::allocator<char> > const&, unsigned int)", referenced from: _main in profile.o ld: symbol(s) not found collect2: ld returned 1 exit status scons: *** [applications/saxs/foxs] Error 1 scons: building terminated because of errors. Any hint appreciated. --Ben Schwarz

2 3

problems with a sample script (basic IMP optimization)
by Benjamin SCHWARZ 12 Aug '10

12 Aug '10

Hi list, As a first step towards a serious usage of IMP, I tried to write a sample code using rigid bodies, restraints and optimizers. With MonteCarlo optimizer, at the end of optimization the model seems to remains unchanged. With ConjugateGradients or SteepestDescent, when using too many steps in the optimizer (under 100 though !?) the script generally crashes with one of these error messages : File "/usr/local/lib/python2.6/site-packages/IMP/core/__init__.py", line 642, in optimize def optimize(self, *args): return _IMP_core.ConjugateGradients_optimize(self, *args) _IMP.UsageException: Attempting to construct a rotation from a non-quaternion value. The coefficient vector must have a length of 1. Got: 0.165858 -0.916284 0.55176 0.327904 gives 1.27905 File "/usr/local/lib/python2.6/site-packages/IMP/core/__init__.py", line 642, in optimize def optimize(self, *args): return _IMP_core.ConjugateGradients_optimize(self, *args) _IMP.UsageException: Attempting to construct a rotation from a non-quaternion value. The coefficient vector must have a length of 1. Got: 0.389486 -0.600546 0.553302 0.99031 gives 1.79921 Moreover when the script finishes, its outputs seem not relevant at all, though the score is better (smaller). maybe it is normal though, due to the choice of my restraints or to the lack of optimization steps… I think my understanding of IMP usage is still very far from perfect, will you help me sorting that out by having a look at my code ? You will fin the script attached to this message, its outlet is really basic : - a structure is loaded ( I choose 1RDT, a complex comprising two protein domains and two peptides) - a map is sampled at 10A for that structure - chains are extracted from the structure, rigidified and randomly moved in space - two restraints are added to the model in order to tale into account - the fit of all atoms to the sampled map - no steric clashes between rigid bodies - optimization thank you very much Dr. Benjamin SCHWARZ Biocomputing group Email : schwarz(a)igbmc.fr Voice : +33 (0)3 68 85 47 30 FAX : +33 (0)3 68 85 47 18 Structural Biology & Genomics Dept. - IGBMC 1 rue Laurent Fries BP 10142 F - 67404 Illkirch CEDEX

4 14

ExcludedVolumeRestraint : meaning of the score
by Benjamin SCHWARZ 09 Aug '10

09 Aug '10

Hi list, I was wondering about the exact meaning of the ExcludedVolumeRestraint score, and how this score was computed. So I took my virtual shovel and had a few hours digging in IMP code, but I still have questions. For what I understood, the score is computed as the sum of harmonic functions over a set of pairs of intersecting spheres (this score being 0.0 for non interacting spheres), but the way these pairs are built is still a little fuzzy for me. Here is the sample python code I wrote using that restraint : rigidChains = {} for chain in IMP.atom.Chains(IMP.atom.get_by_type(mh, IMP.atom.CHAIN_TYPE)) : rigidChains[chain.get_id()] = IMP.atom.setup_as_rigid_body(chain) ... lsc = IMP.container.ListSingletonContainer() for rb in rigidChains.values() : lsc.add_particle(rb.get_particle()) m.add_restraint( IMP.core.ExcludedVolumeRestraint( lsc, IMP.core.LeavesRefiner(refChains.values()[0].get_traits()) ) ) I intuitively expected IMP to consider all pairs of intersecting atom-spheres between any two different chains, but reading the code I could not convince me this is actually what is computed. I could not decide for instance if pairs of atoms in a same chain were considered or not, and in fact I could not even see where exactly atoms were extracted out of my rigid body hierarchies… Could you explain me in a few words if my sample code does what I expect from it, and if not, how to change it to obtain the desired behavior. Thanks a lot Dr. Benjamin SCHWARZ Biocomputing group Email : schwarz(a)igbmc.fr Voice : +33 (0)3 68 85 47 30 FAX : +33 (0)3 68 85 47 18 Structural Biology & Genomics Dept. - IGBMC 1 rue Laurent Fries BP 10142 F - 67404 Illkirch CEDEX

2 5

A couple of naïve questions concerning IMP.Model
by Benjamin SCHWARZ 06 Aug '10

06 Aug '10

Sometimes I load particles in a model that won't be used in the restraints (e.g. molecules loaded twice in order to have a reference for RMSD computations between an initial and an optimized state). I am wondering about the incidence of inserting such "non-useful" particles, is there any overhead ? Also, sometimes the Optimizer appears to need one to provide explicit movements for the particles (RigidBodyMover in the case of Monte Carlo optimizer), and sometimes it appears to know which particles are to be moved (CG). In this latter case, I guess particles that are involved in a Restraint are somehow tagged as such, and the optimizer do not bother with particles that are not tagged as "restraint involved". Am I correct ? Thanks --Ben

2 1

about rotations
by Benjamin SCHWARZ 06 Aug '10

06 Aug '10

Hello list, The information is probably mentioned somewhere but I happened to miss it, so just to be sure can you please confirm that : 1.) IMP rotations (Rotation3) are always origin centered 2.) a RigidBody's transformation is composed of a rotation (about the body center in internal coordinates) and a translation from origin to body center; thus body.get_coordinates() always provides the coordinates of the rigid body mass center. Thank you --Ben

3 2

basic map handling
by Benjamin SCHWARZ 03 Aug '10

03 Aug '10

Hi list, I could not find how to Generate a density map from a structure with a *fixed* dimension : When using IMP.em.SampledDensityMap(ps,resolution,apix) with a fixed size for apix, the grid size depends on the resolution. I would like to write a function that would allow me to generate maps with the same number of vertices (and the same space coverage) for different resolutions. I thus tried : def pdbFile2dmap (pdbStream,densityFile,resolution,apix,sel,bboxCoverage=1.2): ''' pdbStream : where to read the structure densityFile : name of the file where to drop the density map resolution : resolution of the density map apix : voxel size sel : selector for reading structure bboxCoverage : approximate bbox coverage of the density map ''' m= IMP.Model() # read protein mh = IMP.atom.read_pdb(pdbStream,m,sel) # add radius info to each atom IMP.atom.add_radii(mh) # compute bbox, and map size in voxels bbox = IMP.atom.get_bounding_box(mh) diag = bbox.get_corner(0) - bbox.get_corner(1) nx = int(bboxCoverage * diag[0] / apix) ny = int(bboxCoverage * diag[1] / apix) nz = int(bboxCoverage * diag[2] / apix) # compute origin origin = bbox.get_corner(0) + (1-bboxCoverage)/2 * diag dmap = IMP.em.SampledDensityMap() # dmap.CreateVoidMap(nx,ny,nz) ps = IMP.Particles(IMP.core.get_leaves(mh)) dmap.set_particles(ps) dmap.set_origin(origin) dmap.get_header_writable().set_number_of_voxels(nx,ny,nz) dmap.get_header_writable().set_resolution(resolution) dmap.update_voxel_size(apix) dmap.resample() dmap.calcRMS() # computes statistic stuff about the map and insert it in the header print dmap.get_header().show(),"\n" IMP.em.write_map(dmap,densityFile,IMP.em.MRCReaderWriter()) that resulted in : python(10802) malloc: *** mmap(size=18446744073709314048) failed (error code=12) *** error: can't allocate region *** set a breakpoint in malloc_error_break to debug Traceback (most recent call last): File "./pdb2density.py", line 181, in <module> pdbFile2dmap(pdbStream,densityFile,resolution,apix,sel,1.2) File "./pdb2density.py", line 75, in pdbFile2dmap dmap.update_voxel_size(apix) File "/usr/local/lib/python2.6/site-packages/IMP/em/__init__.py", line 428, in update_voxel_size def update_voxel_size(self, *args): return _IMP_em.DensityMap_update_voxel_size(self, *args) MemoryError: std::bad_alloc Can you tell me if my approach is OK, and how to solve my allocation problem ? I would also like to know if there are easy means in IMP to 1.) Strip a density map : out of an existing map, create a new map or restrict the existing one, so as to conserve only a central subset of voxels 2.) Re-sample a map : Conserve the density map "box" size, and change the number of voxels that cover it --Ben

3 7

A remark concerning software design and the get_leaves() function
by Benjamin SCHWARZ 31 Jul '10

31 Jul '10

It is probably a question of taste, but every time I need the leaves of a hierarchy, my intuitive guess is to write mh.get_leaves() instead of IMP.core.get_leaves(mh). So I wonder if there are gory reasons for this function to be outside the class… --Ben

3 2

Questions about example script local_fitting.py
by Benjamin SCHWARZ 21 Jul '10

21 Jul '10

hello list, I started my IMP apprenticeship by trying to run and understand the example script local_fitting.py script. I spent some time rewritting it and parsing the documentation, and I come to you now with remaining questions. I am pretty new to the CryoEM field, so I will probably ask naïve questions. If some of these questions can be answered by reading a manual, please tell me which manual and I will be glad to read it. 1.) As a first remark concerning this script, the module IMP.algebra is used throughout the script, though never imported. Funnily, it puzzles my PyDev plugin but does not appear to lead to execution problems. 2.) I have problems wit regard to the resolution parameter and its apparent absence in density maps : at line 30 the header of a loaded density map is modified. dmap.get_header_writable().set_resolution(resolution) I though, the idea was to fix a possibly distinct resolution value for the resampled map; so I commented the line and replaced it by sampled_input_density.get_header_writable().set_resolution(resolution) at line 36… Which resulted in an error during the local fitting process. I thus checked wether the map header contained a resolution or not : >>> dmap.get_header().get_has_resolution() False >>> dmap.get_header().get_resolution() nan Does the resolution never appear it map files ? What is the incidence/relevance of "arbitrarily" setting this parameter such as it is apparently done in this example ? 3.) I don't fully understand the way objects share elements, and what happens when these elements are moved. For instance, the three objects : line 19 : mh=IMP.atom.read_pdb(IMP.em.get_example_path("input.pdb"),m,sel) line 24 : ps= IMP.Particles(IMP.core.get_leaves(mh)) line 58 : prot_rb = IMP.atom.setup_as_rigid_body(mh) seem to share atoms since when moving : line 60 : IMP.core.transform(prot_rb,local_trans) it looks like every atoms in mh and in ps have also been moved. I thus have the feeling that all three objects share references on the same atoms. If this is true, I don't understand what happens when iteratively applying transformations to prot_rb at the end of the script : line 100 : prot_rb.set_transformation(fitting_sols.get_transformation(i)) line 102 : rmsd=IMP.atom.get_rmsd(IMP.core.XYZs(ps),IMP.core.XYZs(IMP.core.get_leaves(mh_ref))) line 103 : IMP.atom.write_pdb(mh,"temp_"+str(i)+".pdb") Since the transformations are applied iteratively, I would expect them to be composed; hence, parsing from 1 to fitting_sols.get_number_of_solutions() in this order or in reverse order would not lead to the same result. I checked this behavior, and the solutions are (happily) the same whatever the order in which they are parsed... and I can't guess why… 4.) Is there a difference between prot_rb.set_transformation and IMP.core.transform ? 5.) The signature is not the same for function IMP::em::local_rigid_fitting in the documentation and in the script. In the documentation, it has a ( FittingSolutions & fr) argument whereas, according to the script, this in fact seems to be the return type 6.) I could not find the documentation for the class FittingSolutions Thanks for any hint --Ben

5 6