{"id":717,"date":"2010-11-25T15:12:36","date_gmt":"2010-11-25T13:12:36","guid":{"rendered":"http:\/\/www.olafbathke.de\/photography-blog\/?p=717"},"modified":"2010-11-25T15:12:36","modified_gmt":"2010-11-25T13:12:36","slug":"an-implementation-of-an-automated-data-backup-system-under-the-mac-os-x-operating-system","status":"publish","type":"post","link":"https:\/\/www.olafbathke.de\/photography-blog\/2010\/11\/25\/an-implementation-of-an-automated-data-backup-system-under-the-mac-os-x-operating-system\/","title":{"rendered":"An implementation of an automated data backup system under the Mac OS X operating system"},"content":{"rendered":"

This is the first guest post on this blog. It is a great article by Bernd Limbach. Thank you very much Bernd. If you are interested in writing and sharing, please contact<\/a> me:<\/p>\n

A third-party article by Bernd Limbach<\/a>. This is a translation by the author of his original article in German language<\/a>.<\/p>\n

Data backup seems to be a much bigger topic as everybody may think. For the time I have been reading Olaf\u2019s German Blog<\/a> by now, data backup was mentioned at least two times, mostly in the comments section. Here the references to the two German articles:<\/p>\n

https:\/\/www.olafbathke.de\/fotograf-kiel-blog\/2010\/08\/04\/umfrage-wie-gros-ist-dein-datenlager\/<\/a>
\nand
\nhttps:\/\/www.olafbathke.de\/fotograf-kiel-blog\/2010\/04\/14\/cloud-computing-und-bildbearbeitung-was-halten-fotografen-davon\/<\/a><\/p>\n

Out of these two articles everybody can extract these important points: <\/strong><\/p>\n

1) RAID is not a good idea for a backup device.
\n2) External hard drives, DVD\u2019s or similar media can be used as a backup media.
\n3) Redundancy is important.<\/p>\n

These three points should be sufficient to implement a data backup scheme.<\/p>\n

A small excursion to the topic RAID<\/a> :
\nMy Mac Pro is set up with a RAID01 system via the build-in Mac OS X software RAID functionality, which survived the upgrade from 10.5 to 10.6. without any hick-up. Four hard drives are used for this RAID system. There my original RAW files are stored. To me the following points were important for choosing the described set-up:<\/p>\n

– The overall storage capacity of the hard drives should have sufficient capacity to serve me for about 4 years.
\n– There should be a certain protection against a hardware failure of one hard drive.<\/p>\n

Of course this does not ensure the loss of valuable data in case of user mistakes.<\/p>\n

Just as a side note, during my work I joined in to trouble shoot a failed RAID1. One hard drive was defective and the system did not start anymore. Even with support from the manufacturer of that computer we had not been able to rebuilt the RAID1 leaving only the option to reinstall the operating system without a RAID set-up.<\/p>\n

And another hint to think about. Even the same models of hard drives of the very same manufacturers can fail. In theory the purchase at different retailers should reduce the possibility of such a situation. Some people even buy the same size of discs of different manufacturers to reduce the possibility of a hardware failure. I do not have any personal experience with this strategy, so far everything is good in my computer. Everybody needs to make the decision by him- or herself.<\/p>\n

Conclusion for RAID:<\/strong><\/p>\n

It should be treated with certain precautions. As a backup it does not give the data security it may sounds.<\/p>\n

Enough of philosophy, let\u2019s see how a backup strategy can be implemented. For this I did some research for available informations and ideas in the Internet. Out of that collection of information I decided what I wanted to use and sat down to implement them in my system. While I did that for my Mac Pro system, I would expect that under Windows a similar way can be followed. People who implemented a backup scheme under Windows are invited to share their thoughts and actual set-up. I envision the Windows side to be a nice addition to this article! Please leave a comment or contact Olaf.<\/p>\n

Here are my thoughts:<\/strong><\/p>\n

1) The data backup should be done automatically.
\n2) I did not want to use the build-in Time Machine<\/a> .
\n3) Still programs on board should be sufficient to set-up the backup system.
\n4) The file system should be mirrored 1:1, so in the worst case I will be able to copy the folder structure manually.<\/p>\n

To 1) automatically data backup<\/strong><\/p>\n

By nature I am a relatively lazy person, so an automated backup was mandatory. I did not wanted to interfere here.<\/p>\n

To 2) No use of Time Machine<\/strong><\/p>\n

I did not study the technology and features of Time Machine, because I of my thought no. 4. Still, Time Machine may be pretty good in providing a decent backup strategy and includes daily, weekly and monthly backup schemes. It sounds good, but I will not go into more detail here. Maybe someone else uses it, stories are also very welcome!<\/p>\n

To 3) Usage of on board tools of OS X<\/strong><\/p>\n

While Time Machine is an on-board tool, I still will not address it here.<\/p>\n

A few months back I found the following article on the Internet: \u201cTutorial: Backups with Launchd<\/a>\u201d.<\/p>\n

With this I could start to implement my thoughts.<\/p>\n

What is launchd?<\/strong><\/p>\n

Launchd<\/a> is a system to start, stop and manage Daemons, programs and shell-scripts. In Mac OS X it replaced the good old init and various other old structures. For our fellow Windows user just the note that launchd or init is the first program to be started when booting the computer. Only after that the kernel and device drivers will be loaded to be able to use a mouse to navigate the graphical user interface. Let\u2019s just ignore shells or the DOS command prompt here.<\/p>\n

With launchd a lot of tasks can be done, for example specific tasks can be started. Somebody will think, hey, that\u2019s exactly what\u2019s Cron<\/a> for. Yeah, right, I would expect this also can be implemented with cron! I must admit, I do not know any program on board of Windows, which can perform such a task.<\/p>\n

Launchd can also check on the existence or availability of specific Volumes (i.e. a Hard Disc). That\u2019s exactly what I used and what is written in the tutorial. Reading the comments of the tutorial at macresearch.org<\/a> will bring much more inside what to so, so make sure, you read them too!<\/p>\n

Three external hard drives are used for the data backup:<\/p>\n

– PhotoBackup01-10: Backup Disc for the days 1 to 10 of the month.
\n– PhotoBackup11-20: Backup Disc for the days 11 to 20 of the month.
\n– PhotoBackup21-31: Backup Disc for the days 21 to 31 of the month.<\/p>\n

For some of you this seems to be overkill, but Chase Jarvis has two good reads about a backup strategy, which had highly influenced my implementation:<\/p>\n

– How to back up your photography – the basics<\/a> .
\n– Important storage and backup solutions for your photography<\/a> .<\/p>\n

Also have a look at his video<\/a>.<\/p>\n

A minimal aspect related to back-up can be found in the comment of Bayou Bill in the Discerning Photographer\u2019s blog<\/a> dated 6\/30\/2010.<\/p>\n

Well, taking a backup at all is a good and important idea. As some sort of a minimum two separate hard-drives externally connected can satisfactorily serve this. This would match points 2) and 3) from the top of this article.<\/p>\n

Via a shell script (Batch file in DOS jargon) the files will be copied to the backup volumes.<\/p>\n

To copy the files the common command Rsync<\/a> is used, which is embedded in the mentioned shell script at macresearch.org<\/a>. In principal it can be exchanged with a program of similar functionality. Original I wanted to use cpdup<\/a>, which is developed at the DragonFlyBSD project<\/a>. A non existing original port to Mac OS X and my inability to compile a port by myself let me use Rsync. And yes, it also would have been contrary to my statement 3), using only on board tools for implementation. While Rsync can be used to copy files and folders network wide, the backup volumes do not need to be attached to the computer directly. I have not tested that capability.<\/p>\n

Status messages are written in the System Log file. This is not visible directly to the normal user, so access to an administrative account aka root access is mandatory for debugging.<\/p>\n

Beside all my other points of interest and work it took me a couple of weeks to finally figure everything out and get it to run as desired.<\/p>\n

Conclusion:<\/strong><\/p>\n

My implemented of a backup system strategy is not the simplest. A little knowledge of shell commands and scripts as well as launchd are necessary. But all four points of my thoughts could be satisfied. The backup is started automatically, only on board tools without Time Machine are used and the file system, how the files are organized in folders and their hierarchy, could be achieved. At present the backup system is functional, but not in an optimal state. This may change in the future.<\/p>\n

To manage my photos I use Apple\u2019s Aperture 3 (and yes, I did ignore its Vault feature) and after quitting of the application a backup is not automatically started, which would be desirable. As an alternative before shutting down the computer a backup could be run. My best guess is, that AppleScript<\/a> to start a backup after quitting Aperture as well as launchd before the computer shut-down will be able to perform the task. This has not been tried yet.<\/p>\n

If I would not be so lazy to exchange the backup volumes according to their purpose and would bring one backup volume always to a safe place when not in use, then the complete backup strategy would be operational as intended.<\/p>\n

With this I would like to finish this article, which is quite technical. I hope this will inspire some of you to go ahead and finally do a backup. Well, print your photograph, this also is some sort of a backup, right? But I guess, this also will be another story.<\/p>\n

Do you have further hints, tips, similar of different experience with the topic backup, please leave a comment. Olaf and the author will certainly welcome them very much!<\/p>\n

Leaves me only to thank you for your patience to read this lengthy article. Also a big thank you to Olaf for encouragement to publish this article in his blog.<\/p>\n

About the author:<\/strong><\/p>\n

Bernd is a hobby photographer, photographs can be seen on his homepage<\/a> and on flickr<\/a>. Beside shooting he likes to climb and is also a mountaineer, as can be seen from his photographs. To manage stills and also short videos he uses Apple\u2019s Aperture 3, most of the post-processing is done with Aperture as well. Beside Aperture Pixelmator, Photoshop Elements 8 as well as Hugin for panoramic photographs populate his Application folder.<\/p>\n","protected":false},"excerpt":{"rendered":"

This is the first guest post on this blog. It is a great article by Bernd Limbach. Thank you very much Bernd. If you are interested in writing and sharing, please contact me: A third-party article by Bernd Limbach. This is a translation by the author of his original article in German language. Data backup […]<\/p>\n","protected":false},"author":1,"featured_media":0,"comment_status":"open","ping_status":"open","sticky":false,"template":"","format":"standard","meta":{"footnotes":""},"categories":[8],"tags":[],"class_list":["post-717","post","type-post","status-publish","format-standard","hentry","category-how-to"],"yoast_head":"\nAn implementation of an automated data backup system under the Mac OS X operating system - Nature Photography - Olaf Bathke<\/title>\n<meta name=\"robots\" content=\"index, follow, max-snippet:-1, max-image-preview:large, max-video-preview:-1\" \/>\n<link rel=\"canonical\" href=\"https:\/\/www.olafbathke.de\/photography-blog\/2010\/11\/25\/an-implementation-of-an-automated-data-backup-system-under-the-mac-os-x-operating-system\/\" \/>\n<meta property=\"og:locale\" content=\"en_US\" \/>\n<meta property=\"og:type\" content=\"article\" \/>\n<meta property=\"og:title\" content=\"An implementation of an automated data backup system under the Mac OS X operating system - Nature Photography - Olaf Bathke\" \/>\n<meta property=\"og:description\" content=\"This is the first guest post on this blog. It is a great article by Bernd Limbach. Thank you very much Bernd. If you are interested in writing and sharing, please contact me: A third-party article by Bernd Limbach. This is a translation by the author of his original article in German language. Data backup […]\" \/>\n<meta property=\"og:url\" content=\"https:\/\/www.olafbathke.de\/photography-blog\/2010\/11\/25\/an-implementation-of-an-automated-data-backup-system-under-the-mac-os-x-operating-system\/\" \/>\n<meta property=\"og:site_name\" content=\"Nature Photography - Olaf Bathke\" \/>\n<meta property=\"article:published_time\" content=\"2010-11-25T13:12:36+00:00\" \/>\n<meta name=\"author\" content=\"Olaf Bathke\" \/>\n<meta name=\"twitter:label1\" content=\"Written by\" \/>\n\t<meta name=\"twitter:data1\" content=\"Olaf Bathke\" \/>\n\t<meta name=\"twitter:label2\" content=\"Est. reading time\" \/>\n\t<meta name=\"twitter:data2\" content=\"8 minutes\" \/>\n<script type=\"application\/ld+json\" class=\"yoast-schema-graph\">{\"@context\":\"https:\/\/schema.org\",\"@graph\":[{\"@type\":\"WebPage\",\"@id\":\"https:\/\/www.olafbathke.de\/photography-blog\/2010\/11\/25\/an-implementation-of-an-automated-data-backup-system-under-the-mac-os-x-operating-system\/\",\"url\":\"https:\/\/www.olafbathke.de\/photography-blog\/2010\/11\/25\/an-implementation-of-an-automated-data-backup-system-under-the-mac-os-x-operating-system\/\",\"name\":\"An implementation of an automated data backup system under the Mac OS X operating system - Nature Photography - Olaf Bathke\",\"isPartOf\":{\"@id\":\"https:\/\/www.olafbathke.de\/photography-blog\/#website\"},\"datePublished\":\"2010-11-25T13:12:36+00:00\",\"dateModified\":\"2010-11-25T13:12:36+00:00\",\"author\":{\"@id\":\"https:\/\/www.olafbathke.de\/photography-blog\/#\/schema\/person\/8b4c71c9652b64d0fad7bc0ff4de9b94\"},\"breadcrumb\":{\"@id\":\"https:\/\/www.olafbathke.de\/photography-blog\/2010\/11\/25\/an-implementation-of-an-automated-data-backup-system-under-the-mac-os-x-operating-system\/#breadcrumb\"},\"inLanguage\":\"en-US\",\"potentialAction\":[{\"@type\":\"ReadAction\",\"target\":[\"https:\/\/www.olafbathke.de\/photography-blog\/2010\/11\/25\/an-implementation-of-an-automated-data-backup-system-under-the-mac-os-x-operating-system\/\"]}]},{\"@type\":\"BreadcrumbList\",\"@id\":\"https:\/\/www.olafbathke.de\/photography-blog\/2010\/11\/25\/an-implementation-of-an-automated-data-backup-system-under-the-mac-os-x-operating-system\/#breadcrumb\",\"itemListElement\":[{\"@type\":\"ListItem\",\"position\":1,\"name\":\"Home\",\"item\":\"https:\/\/www.olafbathke.de\/photography-blog\/\"},{\"@type\":\"ListItem\",\"position\":2,\"name\":\"An implementation of an automated data backup system under the Mac OS X operating system\"}]},{\"@type\":\"WebSite\",\"@id\":\"https:\/\/www.olafbathke.de\/photography-blog\/#website\",\"url\":\"https:\/\/www.olafbathke.de\/photography-blog\/\",\"name\":\"Nature Photography - Olaf Bathke\",\"description\":\"\",\"potentialAction\":[{\"@type\":\"SearchAction\",\"target\":{\"@type\":\"EntryPoint\",\"urlTemplate\":\"https:\/\/www.olafbathke.de\/photography-blog\/?s={search_term_string}\"},\"query-input\":\"required name=search_term_string\"}],\"inLanguage\":\"en-US\"},{\"@type\":\"Person\",\"@id\":\"https:\/\/www.olafbathke.de\/photography-blog\/#\/schema\/person\/8b4c71c9652b64d0fad7bc0ff4de9b94\",\"name\":\"Olaf Bathke\",\"image\":{\"@type\":\"ImageObject\",\"inLanguage\":\"en-US\",\"@id\":\"https:\/\/www.olafbathke.de\/photography-blog\/#\/schema\/person\/image\/\",\"url\":\"https:\/\/secure.gravatar.com\/avatar\/fc35f3c8d98fb14b513807210379eefaaeea3d014d80ef8c496072e6aed9de03?s=80&d=mm&r=g\",\"contentUrl\":\"https:\/\/secure.gravatar.com\/avatar\/fc35f3c8d98fb14b513807210379eefaaeea3d014d80ef8c496072e6aed9de03?s=80&d=mm&r=g\",\"caption\":\"Olaf Bathke\"},\"sameAs\":[\"https:\/\/twitter.com\/olaf\"],\"url\":\"https:\/\/www.olafbathke.de\/photography-blog\/author\/olaf\/\"}]}<\/script>\n<!-- \/ Yoast SEO plugin. -->","yoast_head_json":{"title":"An implementation of an automated data backup system under the Mac OS X operating system - Nature Photography - Olaf Bathke","robots":{"index":"index","follow":"follow","max-snippet":"max-snippet:-1","max-image-preview":"max-image-preview:large","max-video-preview":"max-video-preview:-1"},"canonical":"https:\/\/www.olafbathke.de\/photography-blog\/2010\/11\/25\/an-implementation-of-an-automated-data-backup-system-under-the-mac-os-x-operating-system\/","og_locale":"en_US","og_type":"article","og_title":"An implementation of an automated data backup system under the Mac OS X operating system - Nature Photography - Olaf Bathke","og_description":"This is the first guest post on this blog. It is a great article by Bernd Limbach. Thank you very much Bernd. If you are interested in writing and sharing, please contact me: A third-party article by Bernd Limbach. This is a translation by the author of his original article in German language. Data backup […]","og_url":"https:\/\/www.olafbathke.de\/photography-blog\/2010\/11\/25\/an-implementation-of-an-automated-data-backup-system-under-the-mac-os-x-operating-system\/","og_site_name":"Nature Photography - Olaf Bathke","article_published_time":"2010-11-25T13:12:36+00:00","author":"Olaf Bathke","twitter_misc":{"Written by":"Olaf Bathke","Est. reading time":"8 minutes"},"schema":{"@context":"https:\/\/schema.org","@graph":[{"@type":"WebPage","@id":"https:\/\/www.olafbathke.de\/photography-blog\/2010\/11\/25\/an-implementation-of-an-automated-data-backup-system-under-the-mac-os-x-operating-system\/","url":"https:\/\/www.olafbathke.de\/photography-blog\/2010\/11\/25\/an-implementation-of-an-automated-data-backup-system-under-the-mac-os-x-operating-system\/","name":"An implementation of an automated data backup system under the Mac OS X operating system - Nature Photography - Olaf Bathke","isPartOf":{"@id":"https:\/\/www.olafbathke.de\/photography-blog\/#website"},"datePublished":"2010-11-25T13:12:36+00:00","dateModified":"2010-11-25T13:12:36+00:00","author":{"@id":"https:\/\/www.olafbathke.de\/photography-blog\/#\/schema\/person\/8b4c71c9652b64d0fad7bc0ff4de9b94"},"breadcrumb":{"@id":"https:\/\/www.olafbathke.de\/photography-blog\/2010\/11\/25\/an-implementation-of-an-automated-data-backup-system-under-the-mac-os-x-operating-system\/#breadcrumb"},"inLanguage":"en-US","potentialAction":[{"@type":"ReadAction","target":["https:\/\/www.olafbathke.de\/photography-blog\/2010\/11\/25\/an-implementation-of-an-automated-data-backup-system-under-the-mac-os-x-operating-system\/"]}]},{"@type":"BreadcrumbList","@id":"https:\/\/www.olafbathke.de\/photography-blog\/2010\/11\/25\/an-implementation-of-an-automated-data-backup-system-under-the-mac-os-x-operating-system\/#breadcrumb","itemListElement":[{"@type":"ListItem","position":1,"name":"Home","item":"https:\/\/www.olafbathke.de\/photography-blog\/"},{"@type":"ListItem","position":2,"name":"An implementation of an automated data backup system under the Mac OS X operating system"}]},{"@type":"WebSite","@id":"https:\/\/www.olafbathke.de\/photography-blog\/#website","url":"https:\/\/www.olafbathke.de\/photography-blog\/","name":"Nature Photography - Olaf Bathke","description":"","potentialAction":[{"@type":"SearchAction","target":{"@type":"EntryPoint","urlTemplate":"https:\/\/www.olafbathke.de\/photography-blog\/?s={search_term_string}"},"query-input":"required name=search_term_string"}],"inLanguage":"en-US"},{"@type":"Person","@id":"https:\/\/www.olafbathke.de\/photography-blog\/#\/schema\/person\/8b4c71c9652b64d0fad7bc0ff4de9b94","name":"Olaf Bathke","image":{"@type":"ImageObject","inLanguage":"en-US","@id":"https:\/\/www.olafbathke.de\/photography-blog\/#\/schema\/person\/image\/","url":"https:\/\/secure.gravatar.com\/avatar\/fc35f3c8d98fb14b513807210379eefaaeea3d014d80ef8c496072e6aed9de03?s=80&d=mm&r=g","contentUrl":"https:\/\/secure.gravatar.com\/avatar\/fc35f3c8d98fb14b513807210379eefaaeea3d014d80ef8c496072e6aed9de03?s=80&d=mm&r=g","caption":"Olaf Bathke"},"sameAs":["https:\/\/twitter.com\/olaf"],"url":"https:\/\/www.olafbathke.de\/photography-blog\/author\/olaf\/"}]}},"_links":{"self":[{"href":"https:\/\/www.olafbathke.de\/photography-blog\/wp-json\/wp\/v2\/posts\/717","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/www.olafbathke.de\/photography-blog\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/www.olafbathke.de\/photography-blog\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/www.olafbathke.de\/photography-blog\/wp-json\/wp\/v2\/users\/1"}],"replies":[{"embeddable":true,"href":"https:\/\/www.olafbathke.de\/photography-blog\/wp-json\/wp\/v2\/comments?post=717"}],"version-history":[{"count":0,"href":"https:\/\/www.olafbathke.de\/photography-blog\/wp-json\/wp\/v2\/posts\/717\/revisions"}],"wp:attachment":[{"href":"https:\/\/www.olafbathke.de\/photography-blog\/wp-json\/wp\/v2\/media?parent=717"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/www.olafbathke.de\/photography-blog\/wp-json\/wp\/v2\/categories?post=717"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/www.olafbathke.de\/photography-blog\/wp-json\/wp\/v2\/tags?post=717"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}