[Step 1] First check if you have required environment setup in your .soft file. My .soft file looks like,
#
# This is the .soft file.
# It is used to customize your environment by setting up environment
# variables such as PATH and MANPATH.
# To learn what can be in this file, use 'man softenv'.
#
#
@bigred
@amber9
@teragrid-basic
@globus-4.0
@teragrid-dev
+mpich-mx-ibm-64
[Step 2] Create condor script including relevant arguments. I put all the required arguments in the "argument" command line of the script. I could get the result with using both batch system and system fork. Don't forget to transfer back the output file. My test script file is the following:
executable = /N/soft/linux-sles9-ppc64/amber9-ibm-64/exe/sander
arguments = -O -i amber_min.in -o min.out.$(Cluster) -c ZINC04273785_ini.crd
-p ZINC04273785_com.top -r ZINC04273785.crd
-ref ZINC04273785_ini.crd
transfer_executable = false
should_transfer_files = yes
when_to_transfer_output = ON_EXIT
transfer_input_files = /home/leesangm/bio/mm_pbsa/amber_min.in,
/home/leesangm/bio/mm_pbsa/ZINC04273785_ini.crd,
/home/leesangm/bio/mm_pbsa/ZINC04273785_com.top,
/home/leesangm/bio/mm_pbsa/ZINC04273785.crd,
/home/leesangm/bio/mm_pbsa/ZINC04273785_ini.crd
universe = grid
grid_resource = gt2 gatekeeper.bigred.iu.teragrid.org/jobmanager-loadleveler
transfer_output_files = min.out.$(Cluster)
error = condorG.err.$(Cluster)
log = condorG.log.$(Cluster)
x509userproxy = /tmp/x509up_u500
queue
Friday, March 7, 2008
Thursday, March 6, 2008
Running AMBER-pbsa on BigRed:[2] Serial-LoadLeveler
IU BigRed loadleveler provides several queues such as DEBUG, LONG, and NORMAL, etc. DEBUG queue(class) has limit of 4hours maximum job cpu time and 15 minutes of maximum processor cpu time. I could run my serial job with DEBUG queue and it took 14 minutes. llclass shows the complete list of queues and description.
Step 1. setup the environment in .soft file
@amber9
+mpich-mx-ibm-64
Step 2. go to the work directory
Step 3. llsubmit serial.job
Step 1. setup the environment in .soft file
@amber9
+mpich-mx-ibm-64
Step 2. go to the work directory
Step 3. llsubmit serial.job
Running AMBER-pbsa on BigRed:[1] Serial-Interactive
Step 1. Setup the environment in .soft file
@amber9
+mpich-mx-ibm-64
Step 2. Go to the work directory and run the following command
$AMBERHOME/exe/sander -O -i amber_min.in -o min.out -c ZINC04273785_ini.crd -p ZINC04273785_com.top -r ZINC04273785.crd -ref ZINC04273785_ini.crd
Step 3. Output file is updated every 2 minutes and it took 20 minutes for me to finish the job completely.
@amber9
+mpich-mx-ibm-64
Step 2. Go to the work directory and run the following command
$AMBERHOME/exe/sander -O -i amber_min.in -o min.out -c ZINC04273785_ini.crd -p ZINC04273785_com.top -r ZINC04273785.crd -ref ZINC04273785_ini.crd
Step 3. Output file is updated every 2 minutes and it took 20 minutes for me to finish the job completely.
Monday, February 18, 2008
Draft of PolarGrid database table design
CREATE DATABASE PolarGrid;
use PolarGrid
# possible entry unit of dataset
# CREATE TABLE Expedition{
# ExpeditionID bigint,
# }
# possible entry unit of dataset
# CREATE TABLE Radar{
# RadarID bigint,
# }
#
# DataChunk
#
# DataChunk is a unit of dataset which is identified by
# (1) spatial information
# (2) temporal information
# (3) triplet of radar information (waveform, transmit antenna, receive antenna)
#
CREATE TABLE DataChunk(
DataChunkID BIGINT NOT NULL AUTO_INCREMENT,
UUID VARCHAR(255),
Desctiption VARCHAR(255),
SamplingFrequency int,
SampleAverage int,
NumberOfWaveform int,
DSPMode VARCHAR(255),
SystemDelay int,
StartPoint point,
StopPoint point,
StartUTC double,
StopUTC double,
Microformat MEDIUMBLOB,
CreationTimestamp timestamp,
RevisionTimestamp timestamp,
PGContactID bigint,
PRIMARY KEY(DataChunkID),
INDEX(StartPoint),
INDEX(StopPoint),
INDEX(StartUTC),
INDEX(StopUTC)
);
#
# FileObject:
#
# FileObject represents minimum unit of dataset. In general
# we assume that this object can be instrumental data or
# output visualization file, or revised data file.
# Please note that WaveformName,TXAntennaName, and RXAntennaName
# are from the file name. There is no validation about this name
# based on the antenna/waveform tables.
#
CREATE TABLE FileObject(
FileObjectID bigint NOT NULL AUTO_INCREMENT,
DataChunkID bigint,
UUID VARCHAR(255),
FileName VARCHAR(255),
RecordTimestamp timestamp,
RadarType VARCHAR(255),
DistributionFormat VARCHAR(255),
WaveformName VARCHAR(255),
TXAntennaName VARCHAR(255),
RXAntennaName VARCHAR(255),
OnlineResource VARCHAR(255),
CreationTimestamp timestamp,
RevisionTimestamp timestamp,
PRIMARY KEY (FileObjectID),
INDEX(DataChunkID),
INDEX(WaveformName),
INDEX(TXAntennaName),
INDEX(RXAntennaName),
INDEX(RecordTimestamp),
);
#
# Waveform
#
# This table defines waveform that transmited between antennas. Each
# radar system can have several different waveforms that it
# can transmit. And that transmitted waveform on that transmit antenna
# can be received on any combinations of antenna. Individual waveform
# describes single waveform that is used by datachunk.
#
CREATE TABLE Waveform(
WaveformID bigint NOT NULL AUTO_INCREMENT,
DataChunkID bigint,
WaveformName VARCHAR(255),
StartFrequency int,
StopFrequence int,
PulseWidth double,
ZeroPiMode int,
PRIMARY KEY (WaveformID),
INDEX(DataChunkID),
);
#
# DataAquisition
#
# This table defines how we describe the setup of antenna.
# This information is included for the waveform and data chunk
# AssociationType field specifies either this setup information is
# used for waveform or data chunk. Similarly, AssociationId field
# specified ID which is exact identity of the item.
#
CREATE TABLE DataAcquisition(
DataAcquisitionID bigint NOT NULL AUTO_INCREMENT,
NumberOfSamples int,
SampleDelay int,
BlankingTime int,
AssociationType VARCHAR(255),
AssociationID bigint,
PRIMARY KEY (DataAcquisitionID),
INDEX(AssociationType),
INDEX(AssociationID)
);
#
# Antenna
#
# This table specifies how we describe the antenna.
#
CREATE TABLE Antenna(
AntennaID bigint NOT NULL AUTO_INCREMENT,
AntennaName VARCHAR(255),
AntennaType VARCHAR(255),
Antennuation int,
AssociationType VARCHAR(255),
AssociationID bigint,
PRIMARY KEY (AntennaID),
INDEX(AssociationType),
INDEX(AssociationID)
);
#
# PGContact
#
# This table specifies contact information.
#
#
CREATE TABLE PGContact(
PGContactID bigint NOT NULL AUTO_INCREMENT,
IndividualName VARCHAR(255),
UNIXLoginName VARCHAR(255),
Email VARCHAR(255),
OrganizationName VARCHAR(255),
PositionName VARCHAR(255),
Voice VARCHAR(255),
Facsimile VARCHAR(255),
Address VARCHAR(255),
OnlineResource VARCHAR(255),
HoursOfService VARCHAR(255),
ContactInstruction VARCHAR(255),
PRIMARY KEY (PGContactID),
INDEX(UNIXLoginName),
INDEX(Email)
);
use PolarGrid
# possible entry unit of dataset
# CREATE TABLE Expedition{
# ExpeditionID bigint,
# }
# possible entry unit of dataset
# CREATE TABLE Radar{
# RadarID bigint,
# }
#
# DataChunk
#
# DataChunk is a unit of dataset which is identified by
# (1) spatial information
# (2) temporal information
# (3) triplet of radar information (waveform, transmit antenna, receive antenna)
#
CREATE TABLE DataChunk(
DataChunkID BIGINT NOT NULL AUTO_INCREMENT,
UUID VARCHAR(255),
Desctiption VARCHAR(255),
SamplingFrequency int,
SampleAverage int,
NumberOfWaveform int,
DSPMode VARCHAR(255),
SystemDelay int,
StartPoint point,
StopPoint point,
StartUTC double,
StopUTC double,
Microformat MEDIUMBLOB,
CreationTimestamp timestamp,
RevisionTimestamp timestamp,
PGContactID bigint,
PRIMARY KEY(DataChunkID),
INDEX(StartPoint),
INDEX(StopPoint),
INDEX(StartUTC),
INDEX(StopUTC)
);
#
# FileObject:
#
# FileObject represents minimum unit of dataset. In general
# we assume that this object can be instrumental data or
# output visualization file, or revised data file.
# Please note that WaveformName,TXAntennaName, and RXAntennaName
# are from the file name. There is no validation about this name
# based on the antenna/waveform tables.
#
CREATE TABLE FileObject(
FileObjectID bigint NOT NULL AUTO_INCREMENT,
DataChunkID bigint,
UUID VARCHAR(255),
FileName VARCHAR(255),
RecordTimestamp timestamp,
RadarType VARCHAR(255),
DistributionFormat VARCHAR(255),
WaveformName VARCHAR(255),
TXAntennaName VARCHAR(255),
RXAntennaName VARCHAR(255),
OnlineResource VARCHAR(255),
CreationTimestamp timestamp,
RevisionTimestamp timestamp,
PRIMARY KEY (FileObjectID),
INDEX(DataChunkID),
INDEX(WaveformName),
INDEX(TXAntennaName),
INDEX(RXAntennaName),
INDEX(RecordTimestamp),
);
#
# Waveform
#
# This table defines waveform that transmited between antennas. Each
# radar system can have several different waveforms that it
# can transmit. And that transmitted waveform on that transmit antenna
# can be received on any combinations of antenna. Individual waveform
# describes single waveform that is used by datachunk.
#
CREATE TABLE Waveform(
WaveformID bigint NOT NULL AUTO_INCREMENT,
DataChunkID bigint,
WaveformName VARCHAR(255),
StartFrequency int,
StopFrequence int,
PulseWidth double,
ZeroPiMode int,
PRIMARY KEY (WaveformID),
INDEX(DataChunkID),
);
#
# DataAquisition
#
# This table defines how we describe the setup of antenna.
# This information is included for the waveform and data chunk
# AssociationType field specifies either this setup information is
# used for waveform or data chunk. Similarly, AssociationId field
# specified ID which is exact identity of the item.
#
CREATE TABLE DataAcquisition(
DataAcquisitionID bigint NOT NULL AUTO_INCREMENT,
NumberOfSamples int,
SampleDelay int,
BlankingTime int,
AssociationType VARCHAR(255),
AssociationID bigint,
PRIMARY KEY (DataAcquisitionID),
INDEX(AssociationType),
INDEX(AssociationID)
);
#
# Antenna
#
# This table specifies how we describe the antenna.
#
CREATE TABLE Antenna(
AntennaID bigint NOT NULL AUTO_INCREMENT,
AntennaName VARCHAR(255),
AntennaType VARCHAR(255),
Antennuation int,
AssociationType VARCHAR(255),
AssociationID bigint,
PRIMARY KEY (AntennaID),
INDEX(AssociationType),
INDEX(AssociationID)
);
#
# PGContact
#
# This table specifies contact information.
#
#
CREATE TABLE PGContact(
PGContactID bigint NOT NULL AUTO_INCREMENT,
IndividualName VARCHAR(255),
UNIXLoginName VARCHAR(255),
Email VARCHAR(255),
OrganizationName VARCHAR(255),
PositionName VARCHAR(255),
Voice VARCHAR(255),
Facsimile VARCHAR(255),
Address VARCHAR(255),
OnlineResource VARCHAR(255),
HoursOfService VARCHAR(255),
ContactInstruction VARCHAR(255),
PRIMARY KEY (PGContactID),
INDEX(UNIXLoginName),
INDEX(Email)
);
Friday, February 15, 2008
PolarGrid database table (initial draft)
This is the basic database table that will directly related to the RSS feeding microformat. Marie and I went through the workflow for generating jpg image and finally agreed on this table design. Thank you so much, Marie!! Still there are some parts which are not that clear to us. As things are clerified, I'll incorperate them to this table.
#CREATE TABLE Expedition{
# ExpeditionID bigint,
#}
CREATE TABLE DataChunk{
DataChunkID bigint NOT NULL,
UUID VARCHAR(255),
Description VARCHAR(255),
SamplingFrequency int,
SampleAverage int,
NumberOfWaveform int,
DSPMode VARCHAR(255),
StartPoint point,
StopPoint point,
StartUTC double,
StopUTC double,
PRIMARY KEY ('DataChunkID')
}
CREATE TABLE FileObject{
FileObjectID bigint NOT NULL,
DataChunkID bigint,
UUID VARCHAR(255),
FileName VARCHAR(255),
RadarType VARCHAR(255),
Timestamp timestamp,
FileType VARCHAR(255),
WaveformName VARCHAR(255),
TXAntennaName VARCHAR(255),
RXAntennaName VARCHAR255).
OnLink VARCHAR(255)
PRIMARY KEY ('FileObjectID')
}
CREATE TABLE Waveform{
WaveformID bigint NOT NULL,
DataChunkID bigint,
WaveformName VARCHAR(255),
StartFrequency int,
StopFrequence int,
PulseWidth double,
ZeroPiMode int,
PRIMARY KEY ('WaveformID')
}
CREATE TABLE DataAcquisition{
DataAcquisitionID bingint NOT NULL,
NumberOfSamples int,
SampleDelay int,
BlankingTime int,
AssociationType VARCHAR(255),
AssociationID bigint
PRIMARY KEY ('DataAcquisitionID')
}
CREATE TABLE Antenna{
AntennaID bigint NOT NULL,
AntennaName VARCHAR(255),
AntennaType VARCHAR(255),
Antennuation int,
AssociationType VARCHAR(255),
AssociationID bigint
PRIMARY KEY ('AntennaID')
}
#CREATE TABLE Expedition{
# ExpeditionID bigint,
#}
CREATE TABLE DataChunk{
DataChunkID bigint NOT NULL,
UUID VARCHAR(255),
Description VARCHAR(255),
SamplingFrequency int,
SampleAverage int,
NumberOfWaveform int,
DSPMode VARCHAR(255),
StartPoint point,
StopPoint point,
StartUTC double,
StopUTC double,
PRIMARY KEY ('DataChunkID')
}
CREATE TABLE FileObject{
FileObjectID bigint NOT NULL,
DataChunkID bigint,
UUID VARCHAR(255),
FileName VARCHAR(255),
RadarType VARCHAR(255),
Timestamp timestamp,
FileType VARCHAR(255),
WaveformName VARCHAR(255),
TXAntennaName VARCHAR(255),
RXAntennaName VARCHAR255).
OnLink VARCHAR(255)
PRIMARY KEY ('FileObjectID')
}
CREATE TABLE Waveform{
WaveformID bigint NOT NULL,
DataChunkID bigint,
WaveformName VARCHAR(255),
StartFrequency int,
StopFrequence int,
PulseWidth double,
ZeroPiMode int,
PRIMARY KEY ('WaveformID')
}
CREATE TABLE DataAcquisition{
DataAcquisitionID bingint NOT NULL,
NumberOfSamples int,
SampleDelay int,
BlankingTime int,
AssociationType VARCHAR(255),
AssociationID bigint
PRIMARY KEY ('DataAcquisitionID')
}
CREATE TABLE Antenna{
AntennaID bigint NOT NULL,
AntennaName VARCHAR(255),
AntennaType VARCHAR(255),
Antennuation int,
AssociationType VARCHAR(255),
AssociationID bigint
PRIMARY KEY ('AntennaID')
}
Wednesday, February 13, 2008
[PG] 80 TB of mobile data
Wednesday, February 6, 2008
[PG]mysql GIS [1] Creating Spatial data and using functions
With MySQL version 5 or higher, you can store GIS data and issue query over it . MySQL provides data types supporting openGIS requirements. Although MySQL does not support full spatial analysis, MBR-based support is very useful, if you want simple bounding box style query. To use this, you don't need any of additional package.
I found useful manual which covers almost everything I was looking for.
http://www.browardphp.com/mysql_manual_en/manual_Spatial_extensions_in_MySQL.html
I found useful manual which covers almost everything I was looking for.
http://www.browardphp.com/mysql_manual_en/manual_Spatial_extensions_in_MySQL.html
Subscribe to:
Posts (Atom)