SD Card component

For Flowcode users to discuss projects, flowcharts, and any other issues related to Flowcode 7.

Moderator: Benj

Post Reply
mnf
Valued Contributor
Valued Contributor
Posts: 1188
Joined: Wed May 31, 2017 11:57 am
Has thanked: 70 times
Been thanked: 439 times
Contact:

SD Card component

Post by mnf »

Just been playing with the SD card component on a data logger shield on an Arduino UNO

I've had a problem:

I can open the SD card (and write to a file etc) - but only in software mode. The card is connected to the SPI bus with chip select on digital 10. Initialise never returns zero with hardware selected - but works aok with (identical settings other than) software mode. Spent a long time looking to fix my code......

Secondly - there is no file close / flush. How do ensure that all my data is written to the card before exiting (or letting the user know card can be pulled etc). I am writing data to the buffer and then moving to next sector (for speed - appendByte seemed slow) - if I use this without forcing a new sector creation will this do the trick?

I have connected a small camera module to the Arduino and hope to write a sequence of files to create time-lapse video. Creating a new file may also have the desired effect?

Martin

User avatar
Benj
Matrix Staff
Posts: 15312
Joined: Mon Oct 16, 2006 10:48 am
Location: Matrix TS Ltd
Has thanked: 4803 times
Been thanked: 4314 times
Contact:

Re: SD Card component

Post by Benj »

Hi Martin,

Usually the problem with AVR and SPI hardware channels is the SS pin is left floating. However as you're using digital 10 as the CS pin this shouldn't be an issue. Might be worth using the component debugger and expose full component tree to modify the properties of the SPI CAL component inside the FAT component. It could be on an AVR the hardware mode is slightly wrong but I have had it working well in the past on AVR so hopefully this isn't the case. It could also be linked to the prescalers for the fast/slow mode. The hardware will likely run a lot faster then software so they might need increasing to match.
Secondly - there is no file close / flush. How do ensure that all my data is written to the card before exiting (or letting the user know card can be pulled etc). I am writing data to the buffer and then moving to next sector (for speed - appendByte seemed slow) - if I use this without forcing a new sector creation will this do the trick?
Yes this sounds fine to me. With something like an OS you need the close/flush commands as programs are time spliced to give the illusion of multi-core functionality on a single processor core. With an embedded system where there is only one task running the close/flush is not required. The only issue is as you say when you want to remove the card.

Very interested to see how you get on with the camera. I've bought several embedded cameras in the past but never been able to dedicate the time to getting them up and running.

mnf
Valued Contributor
Valued Contributor
Posts: 1188
Joined: Wed May 31, 2017 11:57 am
Has thanked: 70 times
Been thanked: 439 times
Contact:

Re: SD Card component

Post by mnf »

Thanks Ben,

Never did get the SD running in hardware mode - although I've managed before, perhaps didn't try the necessary speed combination for the rather slow SD card I was testing with (an old 20x Ultima which had a Rasbian image on and seems to have collapsed to 55MB - but still plenty for some photos!)

I got the code up and running - a couple of problems remain.. The camera I am using is a LinkSprite model (now discontinued but there are several similar looking models on ebay for ~£10) It uses a serial interface (via a level shifter) and encodes the image as a jpg.

1) Resolution seems to be very light dependent - attempting to force the camera to 640x320 only seems to work in good light. In poor light it drops back to 320 x 160 (which is annoying in video sequences)

2) I'd like to time stamp the photo files - but not sure how to convert RTC output to the necessary values for SD Card component? Haven't had time to research this.

3) Another quirk (not really a bug) in Flowcode - I can't define a constant and then use that constant to define another variable (i.e. I can't have a constant BufSize = 64 and then define buffer[BufSize] - which would be a nice feature (it would make tweaking settings easier and less error prone)

4) I currently ignore all the cameras replies (most are of the same format and if you wait they go away!) - but probably should check them for error detection and at startup to check it is ready.

The code would be easy to modify to have (for example) timer controlled photographs (the SD shield I'm using has a RTC), motion detection activated photos, have the Arduino turn on some lights before taking the photo etc....
linksprite camera.fcfx
(33.69 KiB) Downloaded 281 times
I had rather more debug output to the UART - for example if the SD card was retrying but pulled it to make the code smaller (it didn't really - went from ~23k to ~22k) but as it is intended as a stand alone item I'm not sure if it is very useful (maybe lighting a red LED in the event of error - or a green LED whilst writing a photo to SD would be good? or a small LCD?)

Edit:
The commands to set the camera to different resolutions are wrong (I'd downloaded the manual from a different version of the camera).
So in SetResolution: the command for 640 x 480 should be "5600540100"

mnf
Valued Contributor
Valued Contributor
Posts: 1188
Joined: Wed May 31, 2017 11:57 am
Has thanked: 70 times
Been thanked: 439 times
Contact:

Re: SD Card component

Post by mnf »

Scratching my head with this:

Added some simple code to step over existing image files on SD card..

Added code to date/time stamp the files as created (I plan on date / time stamping the images using python on a PC before converting to video). Also some debug output to UART.

Unfortunately this only works for the first file created on each run.. Subsequent files have date time 2/9/1997 and 00:45 etc

Resetting the Arduino gives more of the same.
The debug output "looks" reasonable. So not at all sure what is going wrong here - anyone with some experience of the FAT1 component up for the challenge? (RTC::GetHours seems to be giving 0 (at 21:xx))
linksprite camera.fcfx
(44.81 KiB) Downloaded 268 times
I created date(and time) Stamp variables as globals by mistake - in my original code I had Time GetTimeStamp(() and Date GetDateStamp() in the SetTimeAndDate macro call. The 'link' to maverick-os.dk in the 'hover over' help for SetTimeAndDate seems to be defunct.

Thanks

Martin

User avatar
Benj
Matrix Staff
Posts: 15312
Joined: Mon Oct 16, 2006 10:48 am
Location: Matrix TS Ltd
Has thanked: 4803 times
Been thanked: 4314 times
Contact:

Re: SD Card component

Post by Benj »

Hi Martin,

This is the correct URL for more info on the layout of the time and date info.
http://www.maverick-os.dk/FileSystemFor ... TimeFormat

You need to format the time value like this.
Time = (Hour << 11) + (Min << 5) + (Sec >> 1)

For the Date you would need to format the value like this.
Date = (YearFrom1980 << 9) + (Month << 5) + (Day)

YearFrom1980 = How many years since 1980.
2017 - 1980 = 37

mnf
Valued Contributor
Valued Contributor
Posts: 1188
Joined: Wed May 31, 2017 11:57 am
Has thanked: 70 times
Been thanked: 439 times
Contact:

Re: SD Card component

Post by mnf »

Thanks Ben,

Looks very like the calculation I used to generate the time and date stamp fields.

I'll check again this evening. It seems odd that the first file created has the correct date and time, and subsequent files have the date incorrectly as ../../97

I'm using RTC::GetYear which returns a byte value, what is year 0 for it.?

User avatar
Benj
Matrix Staff
Posts: 15312
Joined: Mon Oct 16, 2006 10:48 am
Location: Matrix TS Ltd
Has thanked: 4803 times
Been thanked: 4314 times
Contact:

Re: SD Card component

Post by Benj »

Hello,

Yes the GetYear just returns the values 0 - 99 so you can probably just add 20 to the value to get the YearFrom1980 value.

mnf
Valued Contributor
Valued Contributor
Posts: 1188
Joined: Wed May 31, 2017 11:57 am
Has thanked: 70 times
Been thanked: 439 times
Contact:

Re: SD Card component

Post by mnf »

Adding 20 to the year did the trick. Not sure why it worked correctly for the first file before.

Ran camera as time lapse and it took ~990 photos of a rather wet and at times misty day. Even so the resultant video clip was pretty good - though the camera doesn't seem to capture green very well (looks rather grey). Using a 20,000mAh battery pack and a 16GB memory card should allow several days recording too.

There are several newer models of the camera available - should work easily (some of the command strings would need modifying - and these should probably be pulled as constants) - which offer benefits of better resolution and weather proofing (hopefully better colour fidelity too) etc.

One function I didn't implement here is changing the camera baud rate - this would be of benefit at higher resolutions - but for my purposes ~1 frame every 40s or so was ideal. I'm not sure I ever found the correct manual for the model I have (given the correct command the command syntax is easy to implement). Most of the commands came from the manual I did find - but some (image size), came from a microPython board program I'd written a few years ago.

Martin
linksprite camera.fcfx
(40.64 KiB) Downloaded 261 times

mnf
Valued Contributor
Valued Contributor
Posts: 1188
Joined: Wed May 31, 2017 11:57 am
Has thanked: 70 times
Been thanked: 439 times
Contact:

Re: SD Card component

Post by mnf »

Revisited the SD card - now I need to read a file...

I can't get OpenFile to work - (locks up).

A couple of points?

ScanFolder returns 2 for file type mismatch IDX = 0 - which filetype? or 3 with IDX = 1.

ReadByteFromScan seems strange - How about ReadFilename(str) - which returns the filename (in a string passed by reference). The byte return value could possibly then indicate if this is the last file or some other info?

OpenFile - crashes - MCU restarts program after ~30s...

This running on Arduino with a 16GB SD card (with two files - one call SUMBURGH.BMP)
sd1.fcfx
(15.58 KiB) Downloaded 257 times
Thanks for any suggestions

Martin

mnf
Valued Contributor
Valued Contributor
Posts: 1188
Joined: Wed May 31, 2017 11:57 am
Has thanked: 70 times
Been thanked: 439 times
Contact:

Re: SD Card component

Post by mnf »

Update:

Running ScanCurrentFolder in a loop with IDX from 0..511 reveals TEST.TXT at IDX 4. This is a file create using the FAT component. This file can be opened and read.

SUMBURGH.BMP (Chosen for its 8.3 filename) doesn't show up and doesn't open (program crashes and restarts)

Changing the name to S1.BMP - file still doesn't open but program doesn't crash (gives a non-zero return from OpenFile). However, it looks like files created on PC don't work. A capacity issue?

Any suggestions ?

Martin

User avatar
Benj
Matrix Staff
Posts: 15312
Joined: Mon Oct 16, 2006 10:48 am
Location: Matrix TS Ltd
Has thanked: 4803 times
Been thanked: 4314 times
Contact:

Re: SD Card component

Post by Benj »

Hi Martin,

IThe ReadByteFromScan is a legacy thing and was done before we had the ability to pass strings around. I've now added a ReadStringFromScan function to the component so you can read the filename/foldername in a single operation,

I've also had a quick look at the cause of the lock up and may have found something. You certainly should be able to open files created from the PC.

Have a go with this component and let us know how you get on.
FAT.fcpx
(67.82 KiB) Downloaded 265 times

mnf
Valued Contributor
Valued Contributor
Posts: 1188
Joined: Wed May 31, 2017 11:57 am
Has thanked: 70 times
Been thanked: 439 times
Contact:

Re: SD Card component

Post by mnf »

Thanks Ben,

Some improvement - but still not quite right...

Initially only 'Arduino' created files appeared in the 'directory' listing created by ScanCurrentFolder.
I reformatted the SD card and created one file ('SUMBURGH.BMP') - this now appears when using ReadStringfromScan as 'SUMBURG.BMP' - note I typed this correctly - it as appearing as a 7.3 rather than 8.3 filename.

Now the open using 'SUMBURGH.BMP' - fails, but doesn't crash.
OpenFile using 'SUMBURG.BMP' - crashes.. Program restarts.

It also seems (very) - slow. Scanning all 512 files using ScanCurrentFolder takes some time!

I've looked through the 'C' code and can see a few possible errors:

For example in FormatFileString:

Code: Select all

    
 while (FCL_I < 11)	   // Fill FAT1_NAME_EXT with spaces - should be 12? (8 + '.' + 3) - although is this necessary at all? (Set FAT1_NAME_EXT[0] = 0)
    {
        FCV_07fa1_FAT1__NAME_EXT[FCL_I] = 0x20;
        FCL_I = FCL_I + 1;
    }
    FCV_07fa1_FAT1__NAME_EXT[11] = 0;	// End of string - should be 12 (8+ '.' + 3). Again is this necessary?
    FCI_TOUPPER(FCL_FILENAME, FCLsz_FILENAME, FCL_FILENAME_COPY,12); // See above - string is max 11 chars so 11 not 12?
    FCL_I = 0;
    while (FCL_I < 8)
    {
 
Not really an error but in FAT1_Initialise:

Code: Select all

   
    FCL_I = 0;
    while (FCL_I < 1)			// Is a loop needed here - is it ever more than 1 iteration?
    {
        FCV_07fa1_FAT1__FOLDER_SIZE[FCL_I] = FCV_07fa1_FAT1__ROOT_SECTORS;
        FCV_07fa1_FAT1__FOLDER_START[FCL_I] = FCV_07fa1_FAT1__ROOT_START;
        FCV_07fa1_FAT1__READ_ADDRESS[FCL_I] = 0;

        FCL_I = FCL_I + 1;
    }
I'll try and look further over the next couple of days (busy day at work here too).

Martin

Post Reply