January 2009 - IMP-dev

documentation build
by Daniel Russel 30 Jan '09

30 Jan '09

Upon Javi's suggestion, we should probably have the online IMP docs updated every night whether or not all the builds and tests success so it tracks, as closely as possibly, the svn state. With so many people contributing to IMP, it is likely that something will be broken most nights, so holding it up based on that makes it less useful. It would be good to also have a release version of the documentation once we have releases.

2 2

benchmarks
by Daniel Russel 30 Jan '09

30 Jan '09

After the discussion of connectivityrestraint performance, I suggest we add a benchmarks folder to IMP. This folder will have C++ programs which time some aspect of IMP so that we can see how speed is doing (I can't figure out how to profile code run from python, which is necessary any time we want to get into running times). Eventually it would be nice to have them run periodically on a reference machine and recorded, but that can wait as it sounds annoying to set up :-)

2 1

Cleanup of config/build/install script
by Daniel Russel 30 Jan '09

30 Jan '09

I just committed a cleanup of the config/build/install scripts following comments and patches posted previously. The changes have the following user-visible consequences: new arguments added to scons: - cxxflags, ldflags which allow the addition of arbitrary compile and link flags - pythonpath which allows dirs to be added to the python path linking has been changed as follows: - on a mac the library ids are now set properly, both for when the libraries are in build/lib and when they are in installed. This means that you can link against them without hassle once installed. It also means that you just have to have PYTHONPATH set properly in order to use IMP - search paths orderings should be consistent everywhere - on linux, unusual directories containing libraries (i.e. things passed in the libflags argument) are added to the libraries runtime search path. This makes it less likely that a library is linked against one external library and run against another, as was easily the case before. imppy.sh has been changed as follows: - the local paths are prepended to the search path, not appended, so local copies of IMP will hide ones elsewhere on the system - external LD_LIBRARY_FLAGS and PYTHONPATH and no longer propagated from the environment (but things can be added to them via scons). This means that "scons test" runs tests in the same environment as if you run the tests yourself with ./bin/imppy.sh - on a mac, LD_LIBRARY_FLAGS is not set as it is no longer needed I have tested everything on my linux box and mac, so I am reasonably confident that they should work. But please report any new problems promptly. And hopefully any issues with other platforms should be cleared up by Monday. Unfortunately, it is still not possible to use the diva copy of modeller as the modeller test scripts are kind of strange and I didn't feel like messing with them. Likewise, since EMBed doesn't have a proper installer, it is still a special case.

1 0

FloatKey In a Double precision?
by Seung Joong Kim 30 Jan '09

30 Jan '09

Dear all IMP developers, By default, Float data type is defined as "typedef float Float;" and it can have a corresponding FloatKey. But in the case of "double" data type for higher accuracy, there is no DoubleKey assigned currently. I'm just curious about it. Do you think is it necessary to define DoubleKey as well? or is it safe to use FloatKey instead? Thanks a lot! Seung Joong Kim

4 5

a note on operator>> and operator<<
by Daniel Russel 29 Jan '09

29 Jan '09

The correct way to have operator<< and operator>> work for reading and writing a class is to declare them as functions in the namespace containing the object. Declaring them as member functions does something else entirely (or absolutely nothing useful, depending on how you do it). The problem is that the compiler searches for operators by checking member functions of the left hand object (for i/o it is the stream itself), then the current namespace and the namespaces containing the arguments. It never looks for member functions of the right hand type. I suggest implementing operator>> as: struct Mine { void show(std::ostream &out=std::cout) const; }; IMP_OUTPUT_OPERATOR(Mine)

1 0

Re: [IMP-dev] [IMP-commits] r1333 - in trunk/modules/em: include src
by Keren Lasker 28 Jan '09

28 Jan '09

Javi please update the SConsript files as well On Jan 28, 2009, at 3:26 PM, Notification of IMP commits wrote: > Author: javi(a)SALILAB.ORG > Date: 2009-01-28 15:26:55 -0800 (Wed, 28 Jan 2009) > New Revision: 1333 > > Added: > trunk/modules/em/include/endian.h > trunk/modules/em/src/endian.cpp > Log: > Some small change to complete ImageHeader class > > Added: trunk/modules/em/include/endian.h > =================================================================== > --- trunk/modules/em/include/endian.h (rev 0) > +++ trunk/modules/em/include/endian.h 2009-01-28 23:26:55 UTC (rev > 1333) > @@ -0,0 +1,61 @@ > +/** > + * \file endian.h > + * \brief functions to deal with endian of EM images > + * \author Javier Velazquez-Muriel > + * Copyright 2007-8 Sali Lab. All rights reserved. > +*/ > + > +#ifndef IMPEM_ENDIAN_H > +#define IMPEM_ENDIAN_H > + > +#include "em_exports.h" > +#include <iostream> > +#include <fstream> > +#include <cstring> > +#include <cstdio> > + > +IMPEM_BEGIN_NAMESPACE > + > +//! Reads from file in normal or reverse order > +/** \ingroup LittleBigEndian > + * \param[in] reverse If true, the data will be read in reverse > order. > + */ > +void reversed_read(void *dest, size_t size, size_t nitems, > std::ifstream& f, > + bool reverse); > + > +//! Writes to a file in normal or reversed order > +/** > + * \ingroup LittleBigEndian > + * > + * This function is the same as fread from C, but at the end there > is a flag > + * saying if data should be read in reverse order or not. > + * > + * \param[in] reverse If true, the data will be read in reverse > order. > + */ > +void reversed_write(const void* src,size_t size,size_t nitems, > + std::ofstream& f,bool reverse = false); > + > + > +//! Conversion between little and big endian. Goes both ways > +/** > + * \ingroup LittleBigEndian > + */ > +#define little22bigendian(x) byte_swap((unsigned char*)& x,sizeof(x)) > +void byte_swap(unsigned char* b, int n); > + > +//! Returns 1 if machine is big endian else 0 > +/** > + * \ingroup LittleBigEndian > + */ > +bool is_big_endian(void); > + > +//! Returns 1 if machine is little endian else 0 > +/** > + * \ingroup LittleBigEndian > + */ > +bool is_little_endian(void); > + > + > +IMPEM_END_NAMESPACE > + > +#endif /* IMPEM_ENDIAN_H */ > > Added: trunk/modules/em/src/endian.cpp > =================================================================== > --- trunk/modules/em/src/endian.cpp (rev 0) > +++ trunk/modules/em/src/endian.cpp 2009-01-28 23:26:55 UTC (rev 1333) > @@ -0,0 +1,85 @@ > +/** > + * \file endian.cpp > + * \brief Calculate score based on fit to EM map. > + * Copyright 2007-8 Sali Lab. All rights reserved. > +*/ > + > +#include <IMP/em/endian.h> > + > +IMPEM_BEGIN_NAMESPACE > + > +void reversed_read(void *dest, size_t size, size_t nitems, > std::ifstream& f, > + bool reverse) > +{ > + if (!reverse) { > + char *ptr = (char *)dest; > + f.read(ptr,size*nitems); > + } else { > + char *ptr = (char *)dest; > + bool end = false; > + for (size_t n = 0; n < nitems; n++) { > + for (int i = size - 1; i >= 0; i--) { > + f.get(ptr + i,1); > + if (f.eof() || f.bad()) { > + end = true; > + break; > + } > + } > + if (end) { > + break; > + } > + ptr += size; > + } > + } > +} > + > +void reversed_write(const void *src, size_t size, size_t nitems, > + std::ofstream& f,bool reverse) > +{ > + if (!reverse) { > + char *ptr = (char *)src; > + f.write(ptr,size*nitems); > + } > + else { > + char *ptr = (char *)src; > + bool end = false; > + for (size_t n = 0; n < nitems; n++) { > + for (int i = size - 1; i >= 0; i--) { > + f.put(*(ptr + i)); > + if(f.bad()) { > + end = true; > + break; > + } > + } > + if (end) { > + break; > + } > + ptr += size; > + } > + } > +} > + > +void byte_swap(unsigned char * b, int n) > +{ > + register int i = 0; > + register int j = n - 1; > + while (i < j) { > + std::swap(b[i], b[j]); > + i++, j--; > + } > +} > + > +bool is_little_endian(void) > +{ > + static const unsigned long ul = 0x00000001; > + return ((int)(*((unsigned char *) &ul))) != 0; > +} > + > + > +bool is_big_endian(void) > +{ > + static const unsigned long ul = 0x01000000; > + return ((int)(*((unsigned char *) &ul))) != 0; > +} > + > +IMPEM_END_NAMESPACE > > _______________________________________________ > IMP-commits mailing list > IMP-commits(a)salilab.org > https://salilab.org/mailman/listinfo/imp-commits

2 1

Re: [IMP-dev] [IMP-commits] r1333 - in trunk/modules/em: include src
by Keren Lasker 28 Jan '09

28 Jan '09

Javi please update the SConsript files as well On Jan 28, 2009, at 3:26 PM, Notification of IMP commits wrote: > Author: javi(a)SALILAB.ORG > Date: 2009-01-28 15:26:55 -0800 (Wed, 28 Jan 2009) > New Revision: 1333 > > Added: > trunk/modules/em/include/endian.h > trunk/modules/em/src/endian.cpp > Log: > Some small change to complete ImageHeader class > > Added: trunk/modules/em/include/endian.h > =================================================================== > --- trunk/modules/em/include/endian.h (rev 0) > +++ trunk/modules/em/include/endian.h 2009-01-28 23:26:55 UTC (rev > 1333) > @@ -0,0 +1,61 @@ > +/** > + * \file endian.h > + * \brief functions to deal with endian of EM images > + * \author Javier Velazquez-Muriel > + * Copyright 2007-8 Sali Lab. All rights reserved. > +*/ > + > +#ifndef IMPEM_ENDIAN_H > +#define IMPEM_ENDIAN_H > + > +#include "em_exports.h" > +#include <iostream> > +#include <fstream> > +#include <cstring> > +#include <cstdio> > + > +IMPEM_BEGIN_NAMESPACE > + > +//! Reads from file in normal or reverse order > +/** \ingroup LittleBigEndian > + * \param[in] reverse If true, the data will be read in reverse > order. > + */ > +void reversed_read(void *dest, size_t size, size_t nitems, > std::ifstream& f, > + bool reverse); > + > +//! Writes to a file in normal or reversed order > +/** > + * \ingroup LittleBigEndian > + * > + * This function is the same as fread from C, but at the end there > is a flag > + * saying if data should be read in reverse order or not. > + * > + * \param[in] reverse If true, the data will be read in reverse > order. > + */ > +void reversed_write(const void* src,size_t size,size_t nitems, > + std::ofstream& f,bool reverse = false); > + > + > +//! Conversion between little and big endian. Goes both ways > +/** > + * \ingroup LittleBigEndian > + */ > +#define little22bigendian(x) byte_swap((unsigned char*)& x,sizeof(x)) > +void byte_swap(unsigned char* b, int n); > + > +//! Returns 1 if machine is big endian else 0 > +/** > + * \ingroup LittleBigEndian > + */ > +bool is_big_endian(void); > + > +//! Returns 1 if machine is little endian else 0 > +/** > + * \ingroup LittleBigEndian > + */ > +bool is_little_endian(void); > + > + > +IMPEM_END_NAMESPACE > + > +#endif /* IMPEM_ENDIAN_H */ > > Added: trunk/modules/em/src/endian.cpp > =================================================================== > --- trunk/modules/em/src/endian.cpp (rev 0) > +++ trunk/modules/em/src/endian.cpp 2009-01-28 23:26:55 UTC (rev 1333) > @@ -0,0 +1,85 @@ > +/** > + * \file endian.cpp > + * \brief Calculate score based on fit to EM map. > + * Copyright 2007-8 Sali Lab. All rights reserved. > +*/ > + > +#include <IMP/em/endian.h> > + > +IMPEM_BEGIN_NAMESPACE > + > +void reversed_read(void *dest, size_t size, size_t nitems, > std::ifstream& f, > + bool reverse) > +{ > + if (!reverse) { > + char *ptr = (char *)dest; > + f.read(ptr,size*nitems); > + } else { > + char *ptr = (char *)dest; > + bool end = false; > + for (size_t n = 0; n < nitems; n++) { > + for (int i = size - 1; i >= 0; i--) { > + f.get(ptr + i,1); > + if (f.eof() || f.bad()) { > + end = true; > + break; > + } > + } > + if (end) { > + break; > + } > + ptr += size; > + } > + } > +} > + > +void reversed_write(const void *src, size_t size, size_t nitems, > + std::ofstream& f,bool reverse) > +{ > + if (!reverse) { > + char *ptr = (char *)src; > + f.write(ptr,size*nitems); > + } > + else { > + char *ptr = (char *)src; > + bool end = false; > + for (size_t n = 0; n < nitems; n++) { > + for (int i = size - 1; i >= 0; i--) { > + f.put(*(ptr + i)); > + if(f.bad()) { > + end = true; > + break; > + } > + } > + if (end) { > + break; > + } > + ptr += size; > + } > + } > +} > + > +void byte_swap(unsigned char * b, int n) > +{ > + register int i = 0; > + register int j = n - 1; > + while (i < j) { > + std::swap(b[i], b[j]); > + i++, j--; > + } > +} > + > +bool is_little_endian(void) > +{ > + static const unsigned long ul = 0x00000001; > + return ((int)(*((unsigned char *) &ul))) != 0; > +} > + > + > +bool is_big_endian(void) > +{ > + static const unsigned long ul = 0x01000000; > + return ((int)(*((unsigned char *) &ul))) != 0; > +} > + > +IMPEM_END_NAMESPACE > > _______________________________________________ > IMP-commits mailing list > IMP-commits(a)salilab.org > https://salilab.org/mailman/listinfo/imp-commits

1 0

performance of Connectivity restraint
by Keren Lasker 28 Jan '09

28 Jan '09

hi all, I am writing a test case for adding hierarchy support in DOMINO, and it seems that the connectivity restraint works incredibly sssllllooowww between two proteins each of ~100 residues. i=0 ub = IMP.core.HarmonicUpperBound(1.0, 0.1) ss= IMP.core.DistancePairScore(ub) r= IMP.core.ConnectivityRestraint(ss) ps = IMP.Particles() ps_refined=[] for j in xrange(2): ps_refined.append(IMP.core.hierarchy_get_leaves(self.h_particles[i+j])) ps.append(self.particles[i+j]) for e in ps_refined: r.add_particles(e) beg = time.time() r.evaluate(None) end = time.time() dt = end - beg print 'connectivity restraint calculation took %9.6f Seconds' % (dt) The evaluate function takes ~20 seconds. am I missing something here? is there a faster implementation using some external library ? thanks, Keren.

2 6

removing indexes
by Daniel Russel 28 Jan '09

28 Jan '09

As was previously warned, I just patched the kernel to remove particle indexes. All the modules in svn are fixed so that they compile except for em, which didn't compile anyway. The changes to be aware of are: - Particles now have names which are strings identifying them (Particle::get_name, Particle::set_name). You should use these when you print particle-related error messages. - Particle::get_index goes away (it is currently left there for backwards compatiblity, but will really go away next week. Domino at least needs to be patched for this). - Model::get_particle(unsigned int) goes away (likewise, it is left in for backward compatibility until next week. The current implementation is linear though.) - the IMP_LIST_IMPL macro takes another argument, so just add a comma at the end if the compiler complains at you

1 0

Re: [IMP-dev] EM issues
by Javier Ángel Velázquez Muriel 28 Jan '09

28 Jan '09

> I can't build EM and so was poking through the code. > > - EM{noise, resample, etc} should be just be {noise, resample, etc} to be > consistent with other names in other modules and the core (all access are > already prefixed with via the paths and namespaces). > > Only the files have that name. Functions don't have the prefix. > > - in general it is less error prone to use an enum for modes rather than a > string (in noise, for example) as they get documented better and can only > exist in correct values > > > > - calling the arguments to add_noise op1 and op2 is horrible. They should > have descriptive names. > > add_noise uses different functions with different meanings for op1 and op2, thus they can't be described properly. They are 2 arguments. > > - given we have a rotation class, "project" should use that rather than its > own convention for defining rotations > > project is a hard function to develop, and the math behind it is far better implemented with a given convention. Hence the definition. > > - in general we avoid passing return return values as arguments > ("project"). At the very least, if you do that you need to make sure that > the non-return arguments are const refs, rather than refs and the last > arguments. > > reasonable > > _______________________________________________ > IMP-dev mailing list > IMP-dev(a)salilab.org > https://salilab.org/mailman/listinfo/imp-dev > > >

2 1