Re: bash script for a FTP dump of the newly create relax wiki -- July 17, 2013 - 14:56

I agree.

I trust that you will know who to share the passwords with,
if some developers are interested. :-)

Best
Troels Emtekær Linnet


2013/7/17 Edward d'Auvergne <edward@xxxxxxxxxxxxx>:
> Hi,
>
> I think, for security reasons, that it would be best not to upload the
> files generated by your two scripts (devel_scripts/wiki_mysqldump.sh
> and devel_scripts/wiki_ftpdump.sh) to a publicly accessible URL.
> Instead, select trusted relax developers can be tasked with running
> cronjobs with these scripts and the private MySQL and FTP user name
> and passwords.  And that the file always kept in confidence.  That
> would solve most of the security problems.  It will also allow the
> wiki to be restored with user information in case of a catastrophic
> failure of the Gigahost infrastructure, or if we migrate to a
> different server in the future.  What do you think?
>
> Cheers,
>
> Edward
>
>
> On 17 July 2013 14:33, Troels Emtekær Linnet <tlinnet@xxxxxxxxx> wrote:
> > Hi Edward.
> >
> > Yes, same scripts should be run together.
> >
> > The mysql database will alter for each page modification.
> > The FTP will only alter on file upload, or if we add extensions.
> >
> > Note. There is a security issue with the mysql dump.
> > If you look for the line:
> >
> > INSERT INTO `user` VALUES
> >
> > The database will contain the fields:
> > `user_id`, `user_name`, `user_real_name`, `user_password`,
> > `user_newpassword`, `user_newpass_time`, `user_email`, `user_touched`,
> > `user_token`, `user_email_authenticated`, `user_email_token`,
> > `user_email_token_expires`, `user_registration`, `user_editcount`
> >
> > Reference:
> > http://www.mediawiki.org/wiki/Manual:User_table
> >
> > And so these fields should be kept secret:
> > `user_password`
> > `user_email`
> > `user_token`
> >
> > The password is hashed, but could be brute force cracked. How, I dont 
> > know, but
> >
> > I suggest complely removing the line in the mysql file, by
> >
> > sed -i '/^INSERT INTO `user` VALUES/d' mysqlfile.sql
> >
> >
> > Best
> > Troels
> >
> >
> > Troels Emtekær Linnet
> >
> >
> > 2013/7/17 Edward d'Auvergne <edward@xxxxxxxxxxxxx>:
> > > Hi Troels,
> > >
> > > For backing up the relax wiki (http://wiki.nmr-relax.com), should I
> > > run both this script and the MySQL dumping script at
> > > http://article.gmane.org/gmane.science.nmr.relax.devel/4163?  Will the
> > > FTP script get the MySQL database contents?  I could create one script
> > > to run both scripts, and then upload using rsync to the relax download
> > > site.  I have now added this script to the relax sources as
> > > devel_scripts/wiki_ftpdump.sh.
> > >
> > > Cheers,
> > >
> > > Edward
> > >
> > >
> > >
> > > On 17 July 2013 13:24, Troels Emtekær Linnet <tlinnet@xxxxxxxxx> wrote:
> > > > Dear relax developers and users.
> > > >
> > > > I hereby send a bash script for a FTP dump of the newly create relax 
> > > > wiki.
> > > > Thereby the wiki can easily be created anywhere else, if created
> > > > together with the mysql database.
> > > >
> > > > To prevent abuse of the webpage hosting service, I need to keep the
> > > > password secret.
> > > >
> > > > The password is shared with Edward d'Auvergne.
> > > >
> > > > I suggest a setup, where Edward run a cronjob a his infrastructure,
> > > > and expose the files to:
> > > > http://download.gna.org/relax/wiki/ftpdump
> > > >
> > > > The FTP dump should only tricker when new files are added.
> > > > That would normally only be when images are uploaded.
> > > >
> > > > Best
> > > > Troels Linnet
> > > >
> > > > -----------------------
> > > > #!/bin/bash
> > > > # -*- coding: UTF-8 -*-
> > > > # Script for dumping ftp server automatically using cron.
> > > >
> > > > # exec test
> > > > exec_test () {
> > > > [ $? -eq 0 ] && echo "-- Command successfully executed" || echo "--
> > > > Command failed; exit 1"
> > > > }
> > > >
> > > > # Filenames
> > > > dbhost="web3.gigahost.dk"
> > > > # The following two fields are kept secret, to prevent abuse.
> > > > # Please contact Troels Linnet at: tlinnet _at_ gmail dot com, for 
> > > > access.
> > > > dbuser="secret"
> > > > dbpwd="secret"
> > > >
> > > > mydate=`date '+%Y%m%d_%H%M'`
> > > > mytime=`date '+%T %Y%m%d'`
> > > > backupfolder="${HOME}/backup/ftpdump"
> > > > backupfoldermirror="${backupfolder}/mirror"
> > > > backupfoldercurrent="${backupfolder}/current"
> > > > mkdir -p $backupfolder $backupfoldermirror $backupfoldercurrent
> > > >
> > > > # Logging
> > > > exec 3>&1 4>&2
> > > > trap 'exec 2>&4 1>&3' 0 1 2 3
> > > > exec 1>>${backupfolder}/ftpdump.log 2>&1
> > > > # Everything below will go to the log file
> > > >
> > > > dump_base () {
> > > > echo "###########################"
> > > > echo "STARTING on: $mytime"
> > > > echo "Mirror syncing..."
> > > > cd $backupfoldermirror
> > > > lftp -e "open ftp://${dbuser}:${dbpwd}@${dbhost} && mirror --no-perms
> > > > --exclude-glob backup_scripts/ --exclude-glob cache/ --exclude-glob
> > > > LocalSettings.php --parallel=10 && bye"
> > > > }
> > > >
> > > > compress_base () {
> > > > echo "Compressing current..."
> > > > DIFF=`diff -q -r $backupfoldermirror $backupfoldercurrent`
> > > > echo -e "Difference between sync and current is:\n$DIFF"
> > > > DIFFARR=($DIFF)
> > > > LENDIFFARR=${#DIFFARR[@]}
> > > > if [ "$LENDIFFARR" -gt "0" ]
> > > > then
> > > > rm -rf $backupfoldercurrent
> > > > cp -p -r $backupfoldermirror $backupfoldercurrent
> > > > tar -zcf ${backupfolder}/ftpdump_${mydate}.tar.gz $backupfoldercurrent/
> > > > fi
> > > > }
> > > >
> > > > dump_base ; exec_test
> > > > compress_base ; exec_test
> > > >
> > > > echo "Done, quit!"
> > > >
> > > > _______________________________________________
> > > > relax (http://www.nmr-relax.com)
> > > >
> > > > This is the relax-devel mailing list
> > > > relax-devel@xxxxxxx
> > > >
> > > > To unsubscribe from this list, get a password
> > > > reminder, or change your subscription options,
> > > > visit the list information page at
> > > > https://mail.gna.org/listinfo/relax-devel