Menu
Index

Contact
LinkedIn
GitHub
Atom Feed
Comments Atom Feed



Tweet

Recent Articles

17/07/2016 15:23
AWS ssh known_host sync
11/07/2016 08:42
File integrity and log anomaly auditing Updated (like fcheck/logcheck)
30/05/2016 13:09
Xenial LXC Container on Debian
12/03/2016 15:33
PHP Zend opcache on Cacti via SNMP
11/03/2016 13:11
Git on (Smart) HTTP with read/write authentication

Glen Pitt-Pladdy :: Blog

Transcend Wi-Fi SD Hacks (CF adaptor, telnet, custom upload)

I've been wanting a wireless way of getting data off cameras onto my workstation for post-processing, but being an older (too good to replace) semi-pro camera it only takes CF, plus added to that the initial Wi-Fi SD devices on the market were heavily locked down (in fact it appears to the point that some can't be used without registration!) with proprietary tools which means you have to accept the vendor dictating to you how you will work or its not going to be much use.

That all changed with the inexpensive Transcend Wi-Fi SD devices entering the market. Not only are these around half the price of many others on the market, but turn out to be running Linux.... and are very hackable as "Pablo" and others discovered. There is even an easy (shell script) hook for executing your own commands / code on the device.

Transcend Wi-Fi SD Card

At last a Wi-Fi SD with openness and flexibility - Well done Transcend!

There is huge potential for customization and adding additional features to this card and little doubt many more pages on this device will be popping up all over the web in the near future.

WARNING!

What I do here may well cause damage, both to kit and to data integrity. A good grasp of Unix fundamentals is a necessity. If you don't understand the whole of this and accept the risks completely then don't attempt any of this.

My approach is based on minimally changing or adding to the device firmware (volatile changes) and making no permanent firmware or configuration changes - all changes are done via hooks executed from the SD storage and deleting the contents of the SD should return the card to "normal" operation.

There is no guarantee however that this goal is achieved since there may be a lot of unknowns lurking still and my experience is that others looking at this card may already have fallen foul of some (see later).

WiFi Signal with SD-CF Adaptor

The first thing is to get the card working in my camera, and that requires a SD-to-CF adaptor. After some research I found an inexpensive one that others seemed to be reporting success with, and it works for me.

SD-CF Adaptor

On putting it all in the camera I got very little signal - in fact it was dropping the connection further than about 1m from the AP. The problems become obvious when you look at it. From a ripped apart Transcend Wi-Fi SD we can see the antenna along to back edge of the card and in the adaptor the antenna will be barely sticking out beyond metal plates either side of it... not going to help:

Wi-Fi SD in SD-CF Adaptor

So the first thing I did is to pry the covers off the adaptor which ensures that the antenna is not shielded (at least on the top):

Wi-Fi SD in Naked SD-CF Adaptor

... and this one is nicely made compared to most stuff I pull apart these days:

Rear Naked SD-CF Adaptor

This has made a massive improvement to the signal and makes it usable now, however since the CF slot is effectively still in a screened cage the signal is far from perfect with the antenna hardly exposed:

SD-CF Adaptor in Camera

None the less I'm getting 50+Mb/s connection speeds in the same room. The next logical step will be to try and put in some means to couple from the SD card antenna to the outside world, but that's for another time.

Setup & Frimware

Out the box I was able to connect to the card via Wi-Fi with a laptop and configure it with the web interface. It powers up at first as an Access Point with the SSID of "WIFISD" and password "12345678", acting as a DHCP Server. After connecting to the card's network, it is accessible on http://192.168.11.254/ with the username and password "admin".

Everything except the upload configuration (eg. to send photos to Facebook) seems to be configurable via the web interface including previewing and downloading photos and videos from the card. For upload configuration you will have to use the mobile app.

"Pablo" and others have already successfully got into the card and have discovered that if present, it will execute autorun.sh on the card at boot. This is a neat hook script that allows a lot of flexibility combined with being able to download a more functional busybox (ARM5) which can be put on the SD and which is mounted below /mnt/sd/ on the card.

My card came with version 1.7 and the mobile app prompted me to upgrade to 1.9 which I was a bit apprehensive in case the nice hooks had been removed, however after upgrading I can confirm the hook is still alive and well. It is worth noting that the upgrade resets the configuration to factory defaults even if the SD data was left intact.

Camera Power Saving

This is mentioned repeatedly in the paperwork with the card. The problem is that by default many cameras use aggressive power saving and shut down after a minute or less idle. Since this card is going to take 30-60 seconds (maybe more) from power up to being connected before it can even start uploading files, you will need to consider this setting on your camera and ensure that the card is kept running long enough to be useful.

The manual / paperwork says to disable power saving (ie. permanently on) but I've had success setting it to several minutes instead.

USB Reader

The packaging states (in find print) "When transferring data from card to computer directly, please only use the included Transcend RDP5 Card Reader."

That is important information worth heeding - I have had many corruptions with other readers. FAT32 is not a clustered filesystem, so in a case like this where multiple systems are accessing the same filesystem, presumably precautions are needed to ensure corruption does not occur.

Shell Access

The first step is to get into the OS, and out the box we already have telnetd available with the shipped busybox so an autorun.sh in the root of the SD card this will give you unauthenticated access:

#!/bin/sh
telnetd -l /bin/bash &

Note: The first line is not necessary with busybox, but I'm in the habit of writing things for maximum portability

After boot you should be able to telnet directly to the device and get straight into a shell.

This is not something that will be wise when off a locked down network with only trusted users as it could result in public access to the contents of your card, but it is a good start for customization of the card.

Intelligent Setup

As well as locking the card down (set a suitable SSID, Wi-Fi key, HTTP password), I've made my card enable services based on the network it finds it's self on. This hooks (appends) into the DHCP client script /etc/dhcp.script which is executed every time a DHCP even occurs (new leases, renewal, disconnect).

I have created a .custom/ directory in the root of the SD with all my scripts & tools and my autorun.sh looks like this:

#!/bin/sh
#telnetd -l /bin/bash &
# based on http://haxit.blogspot.ch/2013/08/hacking-transcend-wifi-sd-cards.html
cp /mnt/sd/.custom/busybox-armv5l /sbin/busybox-extra
chmod a+x /sbin/busybox-extra
# add custom dhcp code (ntpd & local access)
cat /mnt/sd/.custom/ntpd.sh >>/etc/dhcp.script
cat /mnt/sd/.custom/access.sh >>/etc/dhcp.script
# safety - change mount to ro
busybox-extra sed -i.orig -e 's/ -w / /' -e 's/-o iocharset/-o ro,iocharset/' /usr/bin/refresh_sd
refresh_sd
# install the uploader tool
cp /mnt/sd/.custom/autoupload /usr/bin
chmod a+x /usr/bin/autoupload

This assumes you have downloaded the ARM5 busybox binary (see above) and put it in the .custom/ directory, then it appends customization to the DHCP client script, tweaks the refresh_sd script to make the SD mount read-only from the system (as a safety feature) and adds my custom autoupload tool (more about that later).

Customisation - NTP

As I want to use SSL/TLS (https://) to remotely upload files back to my server, I need the card to have a valid clock. By default /etc/init.d/rcS sets the date/time to 201201010000 on boot which invalidates SSL certificates which start later than that.

The busybox-extra added does contain a basic ntpd so we use this in the ntpd.sh snippet (appended to the DHCP client script) with:

# kill existing ntp daemon
[ -f /var/run/ntpd.pid ] && kill `cat /var/run/ntpd.pid`
# try start a new ntp daemon
if [ -n "$ntpsrv" ]; then
    ntpcommand="busybox-extra ntpd"
    for ntp in $ntpsrv; do
        ntpcommand="$ntpcommand -p $ntp"
    done
    $ntpcommand
else
    busybox-extra ntpd -p pool.ntp.org
fi

This will use DHCP provided NTP servers if available else default to the ntp.org pool - change that as you wish.

Customisation - Shell & FTP Access, plus extras

When on a trusted network I want to have shell (and FTP, though the safety of this is suspect for accessint the SD) access to the card. The access.sh snippet (appended to the DHCP client script) looks like this:

# kill existing telnet & ftp daemons
killaccessdaemons() {
    if [ -f /var/run/telnetd.pid ]; then
        kill `cat /var/run/telnetd.pid`
        rm /var/run/telnetd.pid
    fi
    if [ -f /var/run/ftpd.pid ]; then
        kill `cat /var/run/ftpd.pid`
        rm /var/run/ftpd.pid
    fi
}

# start telnet & ftp daemons
startaccessdaemons() {
    if [ ! -f /var/run/telnetd.pid ] || [ ! -d /proc/`cat /var/run/telnetd.pid` ]; then
        telnetd -F -l /bin/bash &
        echo $! >/var/run/telnetd.pid
    fi
    if [ ! -f /var/run/ftpd.pid ] || [ ! -d /proc/`cat /var/run/ftpd.pid` ]; then
        tcpsvd -vE 0.0.0.0 21 ftpd /mnt/sd/DCIM/ &
        echo $! >/var/run/ftpd.pid
    fi
}

# kill the autoupload process
killautoupload() {
    if [ -f /var/run/autoupload.pid ]; then
        kill `cat /var/run/autoupload.pid`
        rm /var/run/autoupload.pid
    fi
}

# start autoupload
startautoupload() {
    if [ ! -f /var/run/autoupload.pid ] || [ ! -d /proc/`cat /var/run/autoupload.pid` ]; then
        autoupload
        echo $! >/var/run/autoupload.pid
    fi
}


# collect info about our surroundings
apssid=`busybox-extra head -n 1 /tmp/iwconfig_maln0.txt | busybox-extra sed 's/^.*ESSID:"\([^"]\+\)".*$/\1/'`
ping -c 1 $router >/dev/null 2>&1
routerMAC=`busybox-extra arp -n $router | busybox-extra awk '{print $4}'`

# check the situation and act accordingly
case "$1" in
    deconfig)
        killaccessdaemons
    ;;
    bound)
        echo "$apssid:$router:$routerMAC" >>/tmp/netid.log
        case "$apssid:$router:$routerMAC" in
            'SSID:RouterIP:RouterMAC')
                # trusted network - run open access (telnetd / ftpd)
                killautoupload
                startaccessdaemons
            ;;
            *)
                # unknown - start auto-uploader TODO
                killaccessdaemons
                startautoupload
            ;;
        esac
    ;;
    renew)
        # do nothing - no change
    ;;
esac

This detects the SSID, the router IP and MAC and appends them together - you need to set the appropriate string in the case statement to match yoru network. For convenience it also logs the strings to /tmp/netid.log where you can check for what it gets on your network.

I can't be certain of FTP being safe for data integrity in this case, but if you want to risk it a simple way of mirroring the data using FTP from an arbitrary path on the the card to the current directory would be to use the lftp tool:

$ lftp -e 'set ftp:passive-mode false; mirror --loop --continue --no-perms SomePath/ .; bye' <WiFi SD IP>

In this case we disable passive mode, but you could set that true if required.

autoupload Tool

This is an experimental tool for mirroring the contents of the SD remotely via HTTP(S) and is intended for on-the-fly backups when running on public networks. It uses a Perl script on the device (yes, like almost everything this also runs Perl!) and relies on a PHP script on a publicly accessible web server to store the files. It would be sensible to secure this with authentication and SSL to avoid abuse.

This is best configured in a sub-directory (say tscardupload/)of a site as index.php for reasons that will soon become obvious. Configure your server with authentication for this directory first, then put the PHP file in renaming it to index.php

Download: PHP Custom Upload Service for Transcend Wi-Fi SD

Now, the trick is that any uploaded file will need to fit within the PHP upload_max_filesize which in turn will need to fit within the PHP post_max_size which in turn will have to fit with the PHP memory_limit (normally OK). If your camera is higher spec and produces larger file sizes then you may have to adjust these limits and by using a sub-directory we can make them specific to that in the Apache config (example includes authentication):

<Directory "/var/www/tscardupload/">
        php_admin_value upload_max_filesize 32M
        php_admin_value post_max_size 32M
        AuthName "Secret Stuff"
        AuthType Basic
        AuthUserFile /etc/apache2/htpasswd-tscardupload
        require valid-user
</Directory>

You will also need to configure the path to store files to and creation modes in the PHP script. Ensure the path is writable by the webserver (eg. owner/group www-data on Debian and derivatives).

Then the Perl script for the SD Card can be added and customized with the curl command and URL appropriate to your site, CA and credentials.

Download: Perl Custom Upload Script for Transcend Wi-Fi SD

After customization you can test this - it should upload any files from DCIM/ below a sub-directory for the authenticated user.

Download Tool (Wi-Fi Tether)

While the FTP option (above) is one (potentially risky) possibility, a bit of fiddling with Firebug, and replacing CGI on the card to capture information and I have all the tools needed for a simple mirroring tool usable when the card is on a local network (directly accessible).

This requires no customization to the device and uses the existing HTTP interface and should run on most Unix style systems (including Mac), possibly only requiring a few additional Perl modules to be installed on the system.

Download: Perl Download (Mirror) Tool for Transcend Wi-Fi SD

This will look in ~/.wifisdMirrorHTTP.conf for credentials and aliases (you get the default one built in so it will need to be overridden here). The format and an example is:

# Lines beginning "#" (comments) and blank lines are ignored
#
# Whit-space separated values
# It is possible to have multiple aliases
# <host or IP> [alias] ... <user>:<pass>
mywifisd.example.com sd sdcard admin:somepassword

The mirror script may need a Proxy setting (optional - see top of script) and flags if the camera time is set to the local timezone (else UTC is assumed) and if filenames should be lower-cased.

Simply run the script with an argument of the hostname (will mirror DCIM/) or specify a path:

$ wifisdMirrorHTTP sd:DCIM/100CAMERA

Things to remember

Among the many pages on the web about hacking this card there are a few things I have discovered that need to be taken into account when dealing with this card any may not have been thought through fully (including with my work):

  • The root filesystem is volatile and any changes appear to only be stored in RAM and will be gone at next boot. This is a good thing for safety, but it means you need to ensure any customization is repeated each time at boot. There is a tiny (nearly full) area storing configuration under /mnt/mtd/ which is a non-volatile (small Flash memory) area.
  • Replacing busybox is a bad idea on this card - the shipped busybox has many additional apps built in that are required for controlling the functionality of the card, hence I put a more functional copy of busybox named busybox-extra instead of breaking functionality by replacing the shipped version
  • FAT32 is not a clustered filesystem and is not safe (will loose data integrity) with concurrent access. There may be some mechanisms being used by the card to solve this, however I have repeatedly corrupted the filesystem when writing to it from both SD and OS. I can't be sure that anything I'm doing will be safe any guarantee data integrity - there is a risk you could corrupt the SD filesystem and loose all your date.
  • Based on the above, shell and FTP access to the card is uncertain for data integrity as I can't be sure it takes due care to avoid corruption - these are not a services offered by the card normally
  • Other services like HTTP mirroring and autoupload scripts may also result in corruption
  • I have so far had no corruptions while in the camera, only in USB SD readers (other than the supplied Transcend one) and USB CF readers via the SD-CF Adaptor. This leads me to belive things are very fragile connecting this card to a PC.

 

Comments:

budi Image  12/02/2014 03:35 :: budi

hello,
i interesting for hack transcend to autoupload files
but how ever i always failure. after install your code my sdcard always could not open in pc. but still run in camera. so i can format the sd card.

could you tell me what is must give more attention that make me failure to set up?

please help
thx
budi

Glen Pitt-Pladdy Image  12/02/2014 23:22 :: Glen Pitt-Pladdy

First, see "Warning" section.

This approach does not attempt to permanently modify the firmware so simply deleting any files added should be sufficient provided you have not corrupted the filesystem integrity.

WiFiSD Image  17/02/2014 01:19 :: WiFiSD

I was wondering if you knew why autorun.sh won't work on two of my WiFi SD cards?

I tried it on 1.9, I was able to downgrade to 1.8 (I couldn't do it to 1.6/1.7) with the same result. I did "chmod +x autostart.sh", I made sure newlines were unix style, but even single line commands won't work.

I tried:
touch /mnt/sd/test.txt - no luck
echo "test" > /mnt/sd/test.txt - nothing
telnet -l /bin/sh & - the telnet port never opens...

The only thing I haven't tried is to boot the SD card on an actual camera but I don think this should matter, I am using a MacBook and tried both internal card readers and the ones that come with the WiFiSD.

Are there other way to access its OS? There were some bugs in 1.6 in the web interface but it was since fixed and I can't downgrade to 1.6

Glen Pitt-Pladdy Image  17/02/2014 22:15 :: Glen Pitt-Pladdy

This is not an easy one to debug since without access to the OS on the device you can't tell what is going on. About all I can suggest is to look at what Pablo found and use a browser to check the /etc/init.d/rcS file where autorun.sh is launched from and if there are any special conditions that may affect you. Also check /tmp/log.rcS which it appears should include a log include the line "run autorun.sh" if it attempts to run it.

Other things worth checking is to verify that the filesystem on the card is consistent (ie. not corrupt) and I'm not sure if things like "Windows format" text might cause problems in these scripts.

All my work has been done on the Unix command line using vi (vim) as the text editor.

pete Image  09/03/2014 15:01 :: pete

Hi I hope u might be able to help me ive lost my password and ssid on my transend sdwifi card and cant get it back to default setting  cheers  Pete

Glen Pitt-Pladdy Image  09/03/2014 15:47 :: Glen Pitt-Pladdy

I think you would be better asking that in support forums. This post is specifically about deeply technical modifications to customize the card. The only thing that may be relevant here is deploying some of the approaches described here to access files like /etc/wsd.conf which contain all the key configuration including passwords.

kg7fu Image  28/03/2014 23:45 :: kg7fu

I'm not sure, but should not $router have a definition, set, or something in access.sh? We get a consistent 'redir error' on the line

ping -c 1 $router >/dev/null 2>&1

Did you actually run this on the card? Because mine fails every time. Also, what do you mean by set the case statement for my network? There are multiple case lines in the script.

Sorry, I'm not a programmer and there's not enough explanation to figure this one out!

Thanks -- jk

Glen Pitt-Pladdy Image  29/03/2014 09:47 :: Glen Pitt-Pladdy

This whole article assumes a basic foundation in Unix (including shell scripting), however this can't just be used as a copy&paste job.

The important thing here is that this shell snippet is appended to the DHCP script which is run everytime the status of DHCP changes. $router and a number of other variables are existing already in that script or are inherited in the environment from the DHCP client that runs that script.

To someone with basic shell scripting knowledge it should be clear which case statement, specifically it's the 'SSID:RouterIP:RouterMAC' bit that is matched with the ones on the other end of the DHCP to verify what network the card is on and how it should behave.

And yes, I do actually use these regularly - in fact just about every photo I've taken since this article the photos have gone via these script. This card is permanently in my camera and I never use a cable connection any more.

Alex J Image  17/04/2014 15:13 :: Alex J

Hey dude, great little write up here. I have a question if you would be so kind : )

if i put this in the auto run:
telnetd -l  /bin/sh &
tcpsvd -E 0.0.0.0 21 ftpd -w /mnt/sd/ &

I get an ftp server which is great. the only problem is that it stops the card from refreshing its contents properly. ie if i have 6 pictures and then remove 3 it still thinks i have 6 on the server. it also breaks the http server in the same way when the ftp is running. any idea if this will be fixable? you seem to have a much better understanding of the works of the thing than me

best
Al

Glen Pitt-Pladdy Image  17/04/2014 18:09 :: Glen Pitt-Pladdy

There are some problems (See "Things to remember" above). One thing is that with more than one device accessing the SD, the caching can get out of sync. If you look at my "autoupload" script you will notice I call /usr/bin/refresh_sd which basically kicks the mountpoint using a mount command, presumably clearing cache to bring the OS into line with the contents of the SD. I'm guessing busybox ftp is not SD aware so doesn't do this Check the filesystem on the SD - chance it's got its self a bit corrupted also.

It appears that the device the SD is plugged into plays a vital role. I'm not sure if there is some sort of locking and/or auto-refresh that can occur triggered by the device. I say this because I've had loads of corruptions with USB SD readers, never with the shipped Transcend reader or my camera. Also I notice with the Transcend reader that OS access to the SD can slow down dramatically (perhaps stall), possibly some lock contention going on or similar?

Arul Image  26/04/2014 15:53 :: Arul

@Glen, thank you so much for this detailed information. I am planing to use this card on my cpap machine so I can remotely copy the data to my linux desktop everyday. Just a quick question--What is the purpose of 1 count of ping in the access.sh? i.e. ping -c 1 $router >/dev/null 2>&1

Glen Pitt-Pladdy Image  26/04/2014 17:21 :: Glen Pitt-Pladdy

The very next line captures the MAC address of the router from the ARP table which is used as part of verifying that we are indeed on our "trusted" network where we can enable unauthenticated telnet etc.

In order for the MAC of the router to be in the ARP table you have to have recently communicated with the device. The ping is really just to ensure recent communication with the router and ensure the MAC address is in the ARP table.

cheche Image  02/05/2014 19:06 :: cheche

How difficult could be to install rsync?

Glen Pitt-Pladdy Image  02/05/2014 19:28 :: Glen Pitt-Pladdy

Well... potentially doing a static build for this architecture and shoving it on the SD would do the job, but there is a catch with these devices. The fundamental problem is that as per the discussion with Alex J above, the SD contains a non-clustered (FAT32) filesystem which we are accessing from two systems both doing their own caching / buffering. At this point I don't fully understand the mechanisms provided for this to work, but there is considerable evidence that applications accessing the data (eg. ftpd, rsync, the web app, my upload script, etc.) in this way need awareness of this to avoid corruption, hence my script calling /usr/bin/refresh_sd for each operation to keep things in sync, and I would guess the web application does as well. A conventional build of rsync will not have that, and my experience is that tools (eg. ftpd) that do not have this awareness are highly prone to causing corruption on this device.

Alexander Image  02/05/2014 22:00 :: Alexander

Could this be used bi-directionally? Ie: to push photos back to the card?

Glen Pitt-Pladdy Image  03/05/2014 08:11 :: Glen Pitt-Pladdy

That's a good question. There does appear to be upload functionality in the shipped CGI, however no means provided in the web interface to use this. This does raise the question if the reason for this is that it might not be safe. If you look at my scripts above, I have tweaked things and forced the SD to be mounted read-only so only the camera can write/delete which minimizes the risks of the corruptions I've seen from two systems accessing the SD concurrently. This is my own extra protection against this (not guaranteed) since I have seen a lot of corruptions with both systems writing to the SD.

Arul Image  03/05/2014 19:06 :: Arul

I have created a write-up/blog page based on all the information provided here with additional steps to setup ssh access (using dropbear) for anyone who is interested in scp'ing files from the card.

http://aselvan.blogspot.com/2014/05/howto-setup-keybased-ssh-scp-to.html

It works great for me so far and I haven't seen any sync issues yet. I was doing a remote execution of the refresh_sd script just before scp'ing files from the card as shown below.

arul@cheetah:/tmp$ ssh root@192.168.xxx.xxx refresh_sd
arul@cheetah:/tmp$ scp -r root@192.168.xxx.xxx:/mnt/sd/DCIM/* .
DSCN0254.JPG                                          100%  836KB 209.1KB/s   00:04

@Glen, once again, thank you so much for sharing this!

Sebastian Image  14/07/2014 10:42 :: Sebastian

I'm trying to send all pictures via curl, if the Card is connected to wifi

autorun.sh
telnetd -l  /bin/sh &
/mnt/sd/chkwlan.sh &

chkwlan.sh
cd /mnt/sd/
for i in 1 2 3
  do
    sleep 4
    #Check Wlan:
      if ifconfig mlan0 | grep -q mlan0 ; then
        echo "#" $i: $(date) online > flist.sh
        #ONLINE
        for pic in `find /mnt/sd -iname "*.JPG"`
          do
            furl="ftp://server/"
            fusr="--user user:pwd"
            fcmd="/bin/curl -T"
            echo $fcmd " " $pic " " $furl " " $fusr >> flist.sh
          done
          /mnt/sd/flist.sh &

          #Leave on success
          break
      fi
  done

The generated Script looks well:
# 2: Sun Jan 1 00:00:12 UTC 2012 online
/bin/curl -T /mnt/sd/DCIM/100DICAM/DSCI0025.JPG ftp://server/ --user user:pwd

I can run this flist.sh without Problems via telnet after startup, but it seems not to be executed on startup. Any help is appreciated!

Glen Pitt-Pladdy Image  14/07/2014 17:55 :: Glen Pitt-Pladdy

The fact that you say this works when run manually suggests that the problem is a race on startup. I suggest you add some debug info into your script to log to a file what it is doing when the card starts up.

I would suspect that 16 seconds (what appears to be the maximum wait in your script) is not going to reliably work at startup. I've seen it take ~2 minutes to establish a connection and a lot depends on what else is going on (eg. if there's IO to the SD then the OS seems to wait / slow down). This is why my scripts are event driven so they trigger when interfaces come up rather than leaving it to chance.

Mark Stam Image  23/09/2014 14:41 :: Mark Stam

Hi,

I also own a Transcend Wifi SD and i can run the autoupload script by telnet but it isnt auto starting when i put on the camera. Is it possible to autostart the script and keeps checking for new files?

Glen Pitt-Pladdy Image  24/09/2014 20:30 :: Glen Pitt-Pladdy

The way of starting the autoupload script described here works is to start it when you are NOT on a known trusted network you have configured. Depending on how you want to use it that may not suit you so it's up to you to tweak the scripts to match how you want to use them.

Jörg Image  03/12/2014 21:45 :: Jörg

Hi,
I've tried to run crond on busybox-extra. After gernerating a /etc/passwd file and a corresponding crontab file crond starts very well. When running the Job I will get an error that crond would not find the script in /var. I've tried some other command. Every command has no Access to the file system. Any idea what to do?!

Thank you for your help

Jörg (Germany)

Glen Pitt-Pladdy Image  04/12/2014 20:11 :: Glen Pitt-Pladdy

What is in the crontab? I suspect that might hold a clue.

Jörg Image  05/12/2014 18:52 :: Jörg

OK, some more informations:
etc/passwd
root:x:0:0:root:/root:/bin/bash

/var/spool/cron/crontabs/root (crontab file)
*/1 * * * * wget "http://xxx.xxx.xxx.xxx:8083/fhem/rss/myrss.jpg -O /mnt/sd/DCIM/info.jpg"

wget works well in telnet, but gives an error in crontab. I've tried without "" and tried '' getting the same error.

I've also tried symlinks:
ln -s /sbin/busybox-extra /sbin/crond
ln -s /sbin/busybox-extra /bin/crontab
rm /bin/vi
ln -s /sbin/busybox-extra /bin/vi
rm usr/bin/wget
ln -s /sbin/busybox-extra /usr/bin/wget

wich works well in telnet, but not with crond

Greetings Jörg and thank you for your help.

Glen Pitt-Pladdy Image  05/12/2014 20:42 :: Glen Pitt-Pladdy

The thing I would check carefully here is what the $PATH is in this instance. I'm not using cron so no first hand experience, but I'd guess that with such a tiny setup as running here there may not be what would be expected on a larger platform and hence cron may be trying to execute wget in an unexpected location. I would suggest you try specifying absolute paths (ie. /usr/bin/wget) in the crontab and see if that helps, or possibly manually set the $PATH.

Jörg Image  05/12/2014 21:02 :: Jörg

wget runs well. When starting crond -f I see that wget reclames an error when trying to write to /mnt/sd/DCIM. This happens only when startet from crond. I've tried to start a script located in /var, but crond will not find this script. Could it be, that cron is running in an capsulated Environment with no Access to the file System?

$PATH = /sbin:/usr/sbin:/bin:/usr/bin

Glen Pitt-Pladdy Image  05/12/2014 22:18 :: Glen Pitt-Pladdy

I've tried this to see if I can debug this problem and am not seeing the same problem. I do notice that your quotes are wrong above - I don't think you actually need any. In my case I grabbed an image from the card it's self (everything I have to hand is SSL) with the crontab:

*/1 * * * * wget http://wifisd.card.address/banner.jpg -O /mnt/sd/DCIM/test.jpg

Then ran crond like this:

# crond -f -d 0

All worked as expected, however I took the precaution of ensuring nothing mounted the SD at this time. As mentioned above, from my experience so far I don't think it is safe to write to the SD from the OS on it. FAT32 is not a clustered filesystem so having it double mounted and writing to it means that corruption is highly likely. Once corrupted I have noticed all sorts of strange behaviour including that the WIFI may fail to connect until the filesystem is fixed / reformatted.

hedehede81 Image  12/01/2015 14:33 :: hedehede81

I start an ftp server via autorun.sh and also telnet client. All is fine. I can upload files and can see them from telnet or web config of Transcend. After powering off and on the MP3 player, it can detect new files (increase file count) but can't play those files. I also tried sync command, remount of sd, same thing.
I take out the SD and put in my PC, I can play uploaded files. I tried comparing files written by SD reader and FTP, everything (owner, permissions etc) except the timestamp is same. (This might be misleading since /mnt/sd is FAT32 and permission dont work?) I tried correcting the date by date -s command, no difference.
What might be the difference of files written by the SD Reader and FTP Server? I also tried the cp command of a playable file, but copy is not playable. only mv command works (which is nothing but the original playable file of course) So any file created by root user/telnet/ftp etc is unplayable on the MP3 player.

Glen Pitt-Pladdy Image  12/01/2015 19:15 :: Glen Pitt-Pladdy

Before I deal with any of this, please see the notes in the article about writing to the SD. In my experience any writing to the SD from the card OS (ie. FTP, cp command etc.) is highly likely to result in a corrupted filesystem. As a result I suspect if you fsck the SD you will find errors on it. When you are messing with things that put data integrity at risk you should expect unpredictable behaviour.

So far as the FTP server, files written will likely have the time on the OS running the service, where files writen by a PC, it's the PC that is writing the files so the time will come from that. Also keep in mind that on this platform are very minimal (busybox) tools which may be missing features in full-blown tools (eg. the "date" command might only be capble of display, not setting). I can tell you that the approach I use above for NTP does set the clock correctly.

hedehede81 Image  13/01/2015 05:54 :: hedehede81

Thank you for your comment. The weird thing is the uploaded files (FTP or cp by telnet) are readable/playable when I plug them to my PC. And there is no corruption. It is just the MP3 player that can't play these files. There should be a minimal difference between playable files and files created by telnet, it can almost play them.  I thought it might the iocharset or shortname flags of the mount option, tried them all, no go. Also tried how Ubuntu mounts: mount -w -t vfat --nosuid --nodev --uid=1000 --gid=1000 --shortname=mixed --dmask=0077 --utf8=1 --showexec --flush /dev/mmcblk0p1 /mnt/sd
didnt work either.Any clue for me?

hedehede81 Image  13/01/2015 13:05 :: hedehede81

OK, I solved my problem, it was the MP3 player, I repartitioned the SD Card as 4gb+28gb and mounted the rest as sd-ext, now everything is fine. (a second FTP server shares sd-ext on port 22) I have one more question though. Is it possible to put DLNA (minidlna) and/or Samba Servers on this card? Any binaries available? It would be awesome if you could put together a tutorial if possible. Many thanks.

Glen Pitt-Pladdy Image  13/01/2015 22:20 :: Glen Pitt-Pladdy

Different implementations of filesystems and playback software may behave very differently when there is any kind of corruption. If there are any corruptions they may prevent things working only under specific conditions - as I said, expect unpredictable behaviour.

It's likely that it's possible to build all sorts of other software for this card. After all there is plenty of storage and it just needs the right cross-compiler. I've not done anything like this, but the OpenWrt project may be a good place to start since it's likely that the OS used here is already a close relative.

Arthur "nibb13" Makhonko Image  10/02/2016 16:04 :: Arthur "nibb13" Makhonko

Hello!

I want to use such card as logging relay. Log data should be written via USB and relayed to some server via wifi connection. My only concern is chip degradation due to frequent writes.
I have solved similar problem in other project by writing to /tmp which is completely in RAM.

Unfortunately i have not such card right now, but i need to know if such way is possible before obtaining one. So, could somebody help me with quick check?

My bet is bind-mounting /tmp to external accessible location. Something like:

mkdir /mnt/sd/tmp
mount -o bind /tmp /mnt/sd/tmp

and to check if "tmp" is accessible via common USB drive access.

Thanks in advance!

Glen Pitt-Pladdy Image  10/02/2016 22:31 :: Glen Pitt-Pladdy

Mounting a filesystem simply maps access to everything below that mountpoint to the storage you mounted, and is only mapped in the OS that mounts the filesystem. In this case let's say OS1 is the Linux running on the SD, and OS2 is a host accessing via USB. Initially both see the SD storage, but as mentioned before, this is not a clustered filesystem so concurrent access will likely end in disaster.

Now if OS1 mounts another filesystem below a directory on the SD storage, then OS1 access gets mapped to that other storage. OS2 is still accessing the SD storage and doesn't know anything about the other storage mounted.

This means that a lot of care is needed when two OS' are accessing the same storage to avoid corrupting the filesystem. You should be able to emulate this situation a number of ways - eg. by sharing an image file with a VFAT filesystem or exporting a block device (or file) with iSCSI containing a VFAT filesystem to multiple hosts.

Arthur "nibb13" Makhonko Image  11/02/2016 02:03 :: Arthur "nibb13" Makhonko

Thanks for answer, Glen!

I think i got it. Mounts (include bind) are handled in kernel and making sense only to machine where kernel is running. External host is communicating with SD via storage controller which can't communicate with running kernel on its own, only being mounted and only by response to requests initiated by kernel. So, despite Linux on SD will have access to bind-mounted dir, external host will see only real dir on this volume (i.e. empty one).

It looks like there are no easy way to render /tmp (RAM-drive) accessible to external host via USB.

Glen Pitt-Pladdy Image  11/02/2016 13:41 :: Glen Pitt-Pladdy

You are absolutely right in your understanding. It is a bit tricky to do that with the filesystem (non-clustered VFAT) and overall design. Logging over a network is often best done with established protocols like syslog which most well designed devices handle, and other like adding WiFi connectivity (take a look at what you can do with OpenWrt on cheap WiFi hardware), but nothing I'm going to be able to help with unless I look at the overall solution in detail ... and I charge for that :-)

Thomas Image  30/05/2016 13:09 :: Thomas

I sincerely hope that you can still be bothered with this old topic :) Using your work as a starting point I wrote a script that uploads all pictures in /mnt/sd/DCIM to an ftp. My problem: This works only for files already present upon booting the card / mounting the sd card file system. New files (e.g. pictures taken with a digital camera with the card inside) are not visible to the system and hence my script. I tried sync and mount -a (in use error) without any success. Any ideas / pointers how to access newly taken pictures?

Glen Pitt-Pladdy Image  30/05/2016 14:02 :: Glen Pitt-Pladdy

If I'm right in what is happening here, this is to be expected. The fundamental thing that comes up in the vast majority of problems people contact me about is that FAT32 is not a clustered filesystem. That makes writing to it from multiple sources almost certain to cause corruption, but also in cases like this while the writing system (camera) is changing the filesystem, a reading system (Linux on the card) is not aware of that so you don't see updates.

If you take a look at my autoupload script you will see that it runs /usr/bin/refresh_sd each time it looks for new files. Essentially that's just a script that unmounts & mounts the filesystem so the kernel refreshes the structures it's caching and hence you should see the new files after. Obviously make sure nothing is keeping the filesystem busy else it will fail. Also keep in mind that cameras could be buffering writes so all the filesystem changes may not be written immediately.

Thomas Image  02/06/2016 13:56 :: Thomas

Thanks for the tip, unmounting and remounting and some sleeping between steps did the trick!

Racusthor Image  10/08/2016 13:06 :: Racusthor

Is all this still possible with firmware 2.3 as I don't get it to work. If not, is downgrading possible and where to find the right firmware?

Glen Pitt-Pladdy Image  10/08/2016 22:44 :: Glen Pitt-Pladdy

Personally I've not tried on anything more than 1.9 and that is completely adequate for my uses. Others are reporting things like the scripts being deleted during boot which means to make it work they have to ensure their scripts also put themselves back in place during boot. What I describe here takes advantage of hooks that the manufacturer has left in place and using them to customise functionality, but if the manufacturer changes their product to not have the hooks, then different approaches will need to be found to add the customisations.

Racusthor Image  11/08/2016 07:07 :: Racusthor

I can't manage to even place a script in the root folder. Any suggestions about that?

Glen Pitt-Pladdy Image  11/08/2016 19:18 :: Glen Pitt-Pladdy

I'm not sure of anything specific that might relate to the version, but not being able to write to parts of the card might also be indicative of corruption of the filesystem which with the non-clustered filesystem is very common if you don't take careful precautions against it. This is a constant theme you will see all through this discussion when writing to the card.




Are you human? (reduces spam)
Note: Identity details will be stored in a cookie. Posts may not appear immediately