WordPress Database Backup is a plugin that I’d relied on in the past to keep a randomly backed up copy of a WordPress site. Unfortunately, that’s not a complete solution as the images or uploads folder, as well as any modified theme pages, won’t be saved during the database backup. Nor will the blog links! Keep an archive copy of everything you put on your website, so everything can be reconstructed in the case of a disaster or even a host switch. In case a website is made up of more than one database, you’d need a different backup solution than what a WordPress plugin can provide.

Use CRON jobs to automate the task of backing up all your hard work. It might take a while to figure things out, but once you do you’ll be secure in the knowledge that you did it and your data is safe.

At first I didn’t have the command part right for the cron job to work. After the time elements you need to indicate the path to sh, any flags (none used here), and the path to the shell script. In this case the command was -

/bin/sh /home/owner/backup/automysqlbackup.sh

where “owner” is the CPanel login name, and “backup” is an empty directory or folder that you create above the public_html directory. Some hosts will have a folder called “etc” already created for you. Check with your host!

In this case the /bin/sh directory was already created at the server level, in other words, I didn’t have to create it. The corresponding first line in the .sh file was -

#!/bin/sh

Originally, the .sh file had an opening #!/bin/bash, but that didn’t work on this particular host.

• A great explanation of CRON commands is found in the rainbow. Nice touch, guys!
• Best explanation of setting up automysqlbackup shell script.
• Get the open source database backup script from sourceforge, automysqlbackup.

The host tells me the script is running successfully, so I know the lack of output is my fault – somewhere. Checks the paths…found it! An error in the path for the configuration file lead to the script trying to run with default parameters, some of them blank. The script ran as far as it could so there was no error reported by the cron daemon to my email address. When the output is successful a log file is sent to that address.

When I finally got the paths straightened out, the backup files were zipped and in their proper places and an email log was sent to the proper address at the time the cron job was to run. It worked! Now I’ll go back and set up a couple of other cron jobs to back up other databases. I’ll use the same script, modified to back up different sets of databases on different schedules.

• If you’re looking for a simple script to make one database backup and save that on your server, check out DaniWeb’s CRON job for backup.
• Backing up your home directory is just as easy!

Once you have your sites backed up automatically, you can do away with any backup plugins that you may have relied on in the past. When at all possible, automate the task of backing up a database. It’ll make your life easier!

I ran into a solution for a problem that I didn’t know I had until I checked to see that data I had imported into an existing MySQL table was imported properly. Data at the beginning of the new table mirrored the CSV nicely, but near the middle of the table I found that certain columns of data were whole numbers, not the decimal figure I was supposed to find.

It’s important to check the format of numerical values. By using the wrong data type when setting up your database you stand to lose precision of your numbers.

For example, when I set up a database I erroneously used the INT data type for a couple of values that needed to show the decimal places. Integers are whole numbers, and therefore, a value like 5.78 in the CSV showed up as 6 in the database table. The software did exactly what I asked it to do, which is a good reason to always verify the data after any manipulations.

The fix was to change the structure of the fields by changing the data ‘Type’ from INT to DECIMAL and specifying the ‘Length/Values’ in the format of ‘N,n’ where N is the total number of digits and n is the number of decimal places. So, for my example of 5.78, I entered “DECIMAL” and “3,2″.

Then, I imported the data again and ta-da! All the data was there in the proper format.

First, we’ll be working with MySQL and to make things easier we’ll use phpMyAdmin to speak to the database.

Second, obtain a delimited text file, which is basically a listing of table columns where each column of data is separated by, or marked off by, or delimited by a character. Common delimiting characters are the comma, semi-colon and quote marks, but any character may be used as long as you indicate what the delimiters are.

In this example the tilde (~) encloses each field of text and the carat (^) denotes the end of the fields.

In phpMyAdmin select your database and table into which you want to import data and click on the Import tab.

Browse to the location of your delimited text file and select the character set of the file if the default is not correct.

Under “Format of imported file”, click CSV, which stands for comma separated values file or character separated values file. An options form will open when CSV is chosen, but if SQL is checked the options form remains hidden.

Under “Options” you can tick off the boxes for ‘Replace table data with file’ and ‘Ignore duplicate rows’ if either function is desired.

Enter the characters that are appropriate for your CSV under ‘Fields terminated by’ and ‘Fields enclosed by’. In the example file above ‘^’ was used for the field termination character and ‘~’ for the field enclosure character.

Characters for escaping fields or terminating lines can be input here, too. Also, if you want to add column names enter the names separated by commas in the blank provided. When ready, hit “Go”.

If successful, you’ll get a message that X number of queries were executed, which means that X number of records were imported into your table. If not successful, check the number of columns and the characters that you’ve input for the CSV and try again.

Your system may limit the size of file that can be imported. If so, check out this LOAD FILE post for a little help.

As always, verify that the data was read into the database correctly. Check that the data for each column of the table matches up with the source delimited text file.

Instead of trying to convert the Access files to MySQL I chose to work with delimited text files that were available. Ten text files were used to represent ten tables for the database. It was easy enough to read in the delimited text files into OpenOffice Calc by identifying the delimiting characters and importing the new .csv file into MySQL, but one file in particular was really big and presented a problem.

I tried to work with the big text file in Notepad and HTML-Kit, but it was too cumbersome due to the file size – over 500,000 records! Not enough copy and paste operations could be done in a day to make me want to go through it again when the original database gets updated and I have to convert the scary big text file into MySQL format again. Knowing that the original database can be updated at any time, I needed a way to get that huge file into MySQL with a minimum of effort.

After looking around a bit, I downloaded a piece of conversion software called Full Convert from SpectralCore.

Installation went smoothly and I noted there was an uninstall feature in case the software wasn’t a good fit for my needs. It seemed promising in that there are several databases that can be converted back and forth, but the function to convert a text file to MySQL tables did not work for me. The text file used the characters ‘^’ to terminate fields and ‘~’ to enclose fields. However, the software did not give a choice for the field-enclosing character.

Since it didn’t work for me I uninstalled it, but the Full Convert software may be promising if you’re moving files from one database system to another. Here’s the list of DBs they work with:

Supported source databases
- Microsoft Access, dBase, FoxPro, Microsoft Excel documents, Interbase / Firebird, Lotus 1-2-3, MySQL, Oracle, PostgreSQL, ODBC sources, Paradox, SQL Server, Delimited text files, XML

Supported target databases
- MySQL, SQL Server, Oracle, PostgreSQL, Access, Interbase/Firebird

Further digging lead me to a solution for this problem of not being able to import a really big, delimited text file into a table in MySQL.

LOAD DATA INFILE

The solution is to read the file directly into MySQL using the MySQL statement “LOAD DATA INFILE”.

For my situation the code that worked was this -

LOAD DATA INFILE ‘data.txt’ INTO TABLE databasename.tablename FIELDS TERMINATED BY ‘^’ ENCLOSED BY ‘~’;

Don’t forget the trailing semi-colon and don’t use single quotes around the database and table names (and leave the semi-colon in the space provided for the Delimiter).

How did I find my solution? Searching at WebmasterWorld for “convert text to mysql” did the trick.

A simple post by a fellow developer that was written to help someone else out a couple years ago helped lead me to the LOAD_DATA function. Thanks, dreamcatcher!

I considered writing my own php script where I would have separate pull-down boxes for the day, month and year to assure that there was no mistake as to the actual dates. That would take a little extra format massaging to create the date in MySQL format, so I looked for scripts that others had already written. Who wouldn’t?

Formatting dates can drive a person crazy. Does ’06-08-2000′ mean June 8th or the 6th of August? Writing for the Web should be universal. The month-day format that we tend to use in the U.S. is confusing to everybody else who uses the day-month format, and vice-versa.

Writing by hand I like to use this format ’12-Mar-2000′ so there is no ambiguity, but that format needs to be converted to ‘YYYY-MM-DD’ format when sending dates to MySQL.

So, what are my requirements for such a Date-Picker Script?

• easy, intuitive for the user
• easy for me with output in MySQL format (YYYY-MM-DD)
• unrestricted license would be nice

I tried a few different date picker scripts and decided upon SpiffyCalendar because it’s the most advanced, yet easily modified to suit my purposes. Also, there is an established support system via a Yahoo! user group. That’s really nice, and unexpected.

Here’s the scripts I tried and the pros and cons for each…most weren’t as user-friendly as SpiffyCalendar.

PHP Calendar

PROS
• I like the look of this calendar, but it’s difficult to determine how to modify the calendar other than for css of calendar appearance.
• Date output is already in MySQL format: YYYY-MM-DD

CONS
• Can’t select year from calendar unless you click back or forward to the actual month.
• Ugly and excessively long query strings in links.

DatePick

PROS
• I like the little calendar icon.

CONS
• Can’t select year from calendar unless you click back or forward to the actual month.
• Timestamp is included. Don’t need it.
• Relies on javascript and pop-up window to select date.
• Calendar doesn’t update well when moving from month to month. Part of the calendar doesn’t get loaded in so all days are not selectable. Have to manually widen window to see all days.
• Can’t select year directly, have to click through months to get to next year.
• No directions on how to modify output.

DateTimePick

PROS
• I like the little calendar icon.
• Can modify output via provided directions.

CONS
• Can’t select year from calendar unless you click back or forward to the actual month.
• Relies on javascript and pop-up window to select date.
• Calendar doesn’t update well when moving from month to month. Part of the calendar doesn’t get loaded in so all days are not selectable. Have to manually widen window to see all days.
• Can select year directly, but calendar doesn’t update well.

SpiffyCalendar

Sign up to the Yahoo! Group to download the script. Script used to be offered at the now defunct GeoCities. The above link is the first place I found it posted afterwards.

PROS
• Calendar pops into page, not a separate window.
• Can modify output via provided directions.
• Can get help at Yahoo! Groups.
• Unrestricted license.
• Can refresh page containing calendars as link is not modified after picking a date or two.
• Highlights holidays, although will have to remove Dec 1st developer’s birthday. (OK, modified holiday array in script on lines 86-87.)

CONS
• Format of date will have to be modified for MySQL. (OK, Added "cal1.dateFormat="yyyy-MM-dd"; after instantiation in header for calendar named ‘cal1′.)
• Relies on javascript.
• Had to modify the images folder location in the .js file to an absolute address, although things worked fine on my local server with a relative address.

I’m sure there are plenty of these date-picker scripts floating around the Web. Do you have a different one to share with us? Leave a comment!

It turns outs that the numeric type that you select for your database variables matters greatly – DUH!

I had an occurrence last week where a client entered values over 200 in a form, yet when the numbers were stored in the MySQL database were capped at 127. At first the error wasn’t spotted but when multiple entries were reported with the same 127, it became obvious that something was wrong.

My error was using tinyint instead of smallint as the numeric data type when setting up the MySQL table.

The MySQL Manual clearly gives us limits for each data type, as found in the following table:

Numeric Data Types in MySQL

I don’t think we’ll have an occasion to use BIGINT, but the other data types will definitely be of use.

When all else fails, RTFM!

SELECT statements were used for the most part which combined data from up to nine different mysql tables for the most inclusive report. A problem surfaced in that the reports did not show all the members. Some of the newer members did not yet have accomplishments that would show up in the output. Instead of seeing blank values for the newbies, the output did not list those new people at all. The reports were incomplete without listing all the members.

It would be helpful to see those blanks in the report as a reminder that some of the new people still need to satisfy certain requirements.

LEFT JOINS to the rescue!

LEFT JOINs are used to combine data from several database tables that may or may not have content pertaining to all members. Using the LEFT JOIN having the members table on the left will retain all members in the final report whether or not they have entries in the associated tables.

Instead of using the regular construct, SELECT data-col1, data-col2, data-col3 FROM table1, table2 WHERE condition1 = condition2, a LEFT JOIN was used like so:

SELECT data-col1, data-col2, data-col3 FROM table1 LEFT OUTER JOIN table2 ON condition1 = condition2

where the two table names are separated by “LEFT OUTER JOIN” and the WHERE clause is changed to the ON condition.

Emphasis is given to the table named on the left, table1, so that all its rows will be retained through the SELECTing of columns even though null values are encountered.

Original code that produced a report without blank lines:

$fems = mysql_query(“SELECT `last_name`, `first_name`, MAX(`certs_ID`) AS mcrt, `fema_date`, `pin` FROM `members`, `fema`, `fema_attend`, `certs`, `sar-certs` WHERE `pin` = `mem_pin` AND `fema_ID` = `fema_ids` AND `member_pin` = `pin` AND `cert_ID` = `certs_ID` GROUP BY `last_name`, `first_name` “)

Updated code using LEFT JOINS that produced a complete report that contained blank lines:

$fems = mysql_query(“SELECT `last_name`, `first_name`, MAX(`certs_ID`) AS mcrt, `fema_date`, `pin` FROM `members` LEFT OUTER JOIN `fema_attend` ON `pin` = `mem_pin` LEFT OUTER JOIN `fema` ON `fema_ID` = `fema_ids` LEFT OUTER JOIN `sar-certs` ON `member_pin` = `pin` LEFT OUTER JOIN `certs` ON `cert_ID` = `certs_ID` GROUP BY `last_name`, `first_name` “)

You’ll need a new LEFT JOIN for each table that you want to tack onto the SELECT statement in order to keep the blank values in the output.

Keep smiling and keep reading!

Here’s the queries and partial output in question:

$fems = mysql_query(“SELECT `last_name`, `first_name`, MAX(`certs_ID`) AS mcrt, `fema_date`, `pin` FROM `members` LEFT OUTER JOIN `fema_attend` ON `pin` = `mem_pin` LEFT OUTER JOIN `fema` ON `fema_ID` = `fema_ids` LEFT OUTER JOIN `sar-certs` ON `member_pin` = `pin` LEFT OUTER JOIN `certs` ON `cert_ID` = `certs_ID` GROUP BY `last_name`, `first_name` “) or die (“Query failed here 12″);
while ($fem = mysql_fetch_array($fems)) {
echo “<td>$fem[last_name]</td>”.” <td>$fem[first_name]</td> “;

$high_cert = mysql_query(“SELECT `cert` FROM `certs` WHERE `cert_ID` = $certif[mcert]“);
while ($high_cer = mysql_fetch_array($high_cert)) {
echo “$high_cer[cert]“;
}…

And, here’s part of the ugly report that code section produced:

What’s happening here to produce the error is that one of the queries produces some null values for the newest members of the organization who haven’t completed some requirements as of yet.

While fetching values from the $fems array an error is dumped on the screen that states:
Warning: mysql_fetch_array(): supplied argument is not a valid MySQL result resource in C:\xampp\htdocs\org_member.php on line 766

How did I figure out what that error meant? Using the tool phpmyadmin! Once you open phpMyAdmin and have selected your database, click on the SQL tab.

I copied and pasted the first SELECT statement in the SQL box that is labeled “Run SQL query/queries on database psar-data:”, where psar-data is the name of my database.
SQL query inside phpMyAdmin.

The page that is returned after pressing Go shows the SQL query and its output that contains NULL values.

Now, when the data set was subjected to the second query the null values produced the invalid result resource warning error. The second query was this:

$high_crts = mysql_query(“SELECT `cert` FROM `certs` WHERE `cert_ID` = $fem[mcrt]“);
while ($high_crt = mysql_fetch_array($high_crts)) {
echo “$high_crt[cert]“;
}

Outputting values to the screen from this query would be so much nicer if the error would go away and just leave a blank. This example is a perfect use for the @ operator, or PHP error suppression operator. When placed in front of a PHP function the @ operator does not output error statements and mess up your data reports.

In the case where null values are encountered by the PHP function mysql_fetch_array the ugly warning error is produced. Placing the @ operator directly in front of mysql_fetch_array, like so @mysql_fetch_array, gives the output we’re seeking.

Yes, that output looks more professional without the error message and shows exactly what we want. In this case the blanks are meaningful and show exactly where a member needs to complete certain requirements.