Today, I thought we’d do a followup post to this one where we initially started looking at shell scripts. Exports are by default, single-threaded, i.e., you get 1 file exported out and when it hits the maximum (2 GB in most cases), a separate file is spawned. But what if we wanted to do multi-threaded exports? MaxL allows you to do this and it reduces the time needed to export out large databases. To do this with our shell script structure, let’s make a simple update to our definition file, by adding one more column at the end.
The third column, is intended to act as a thread count. In other words, the “CUBE” database should export to 1 file, while “CUBE2” should export 2 files. Now I’ve also made a copy of the original MaxL file from earlier.
The second one is identical except that it exports 2 threads:
export database $2.$3 Level0 data to data_file "$5//$2_$3_1.txt","$5//$2_$3_2.txt";
We also need to change up the “while” loop in our shell script.
while IFS=',' read -r APP DB THREADS; do
sh $MAXLSH/startMaxl.sh -D $MAXLDIR/Nightly_Export_$THREADS.maxls $KEY $ESS_SERVER $APP $DB $LOGDIR/Essbase_Daily $HOMEDIR/Export/ESSBASE_LEV0_DATA/${DATESTAMP}_Export 2>&1 >> $ESSBASE_DAILY_LOG
done < $DEFDIR/ESSBASE_BACKUP_2.txt
We’ve added a “THREADS” parameter which tells the script which MaxL file to call. Now if we run the script again, we get our desired output.
The script in it’s entirety looks like this:
#!/bin/sh
## Set variables
HOMEDIR=/App/Admin
DEFDIR=$HOMEDIR/Definitions
LOGDIR=$HOMEDIR/Logs
ERRORDIR=$HOMEDIR/Errors
MAXLDIR=$HOMEDIR/MAXL
DATESTAMP=`date +"%Y_%m_%d"`
ESSBASE_DAILY_LOG=$LOGDIR/${DATESTAMP}_Essbase_Daily.log
APPDIR=/App/Oracle/Middleware/user_projects/epmsystem1/EssbaseServer/essbaseserver1/app
MAXLSH=/App/Oracle/Middleware/user_projects/epmsystem1/EssbaseServer/essbaseserver1/bin
KEY=213367771,507947599
ESS_SERVER=UNLK.CUBE.COM
## Create directories
echo "*******************************************************" >> $ESSBASE_DAILY_LOG
echo Essbase Backup Started on `date` >> $ESSBASE_DAILY_LOG
echo "*******************************************************" >> $ESSBASE_DAILY_LOG
echo -e '\n' >> $ESSBASE_DAILY_LOG
mkdir -pv $HOMEDIR/Export/ESSBASE_LEV0_DATA/${DATESTAMP}_Export >> $ESSBASE_DAILY_LOG
echo -e '\n' >> $ESSBASE_DAILY_LOG
echo "Export process begins." >> $ESSBASE_DAILY_LOG
echo -e '\n' >> $ESSBASE_DAILY_LOG
## Loop through and create exports
while IFS=',' read -r APP DB THREADS; do
sh $MAXLSH/startMaxl.sh -D $MAXLDIR/Nightly_Export_$THREADS.maxls $KEY $ESS_SERVER $APP $DB $LOGDIR/Essbase_Daily $HOMEDIR/Export/ESSBASE_LEV0_DATA/${DATESTAMP}_Export 2>&1 >> $ESSBASE_DAILY_LOG
done < $DEFDIR/ESSBASE_BACKUP_2.txt
echo -e '\n' >> $ESSBASE_DAILY_LOG
echo "Export process ends." >> $ESSBASE_DAILY_LOG
echo -e '\n' >> $ESSBASE_DAILY_LOG
echo "*******************************************************" >> $ESSBASE_DAILY_LOG
echo Essbase Backup Ended `date` >> $ESSBASE_DAILY_LOG
echo "*******************************************************" >> $ESSBASE_DAILY_LOG
echo -e '\n' >> $ESSBASE_DAILY_LOG
exit
And there you have it. A simple example of how to use shell effectively to keep our scripts lean. As you can imagine, this type of approach can be helpful in a number of scenarios. More parameters could be added to the definition file to control other aspects of your maintenance process.
