Forum Index
HomeZBasic Home   Forum RulesForum Rules   Forum FAQForum FAQ   MemberlistMemberlist   UsergroupsUsergroups   RSS FeedRSS Feed
Site SearchSite Search   LinksLinks   DownloadDownload   Digests and SubscriptionsDigests and Subscriptions
ProfileProfile   Log in to check your private messagesLog in to check your private messages   Log inLog in   RegisterRegister
serial port, tasks, GPS , and average processing power

 
Post new topic   Reply to topic    Forum Index -> ZBasic Language
Author Message
spamiam



Joined: 13 Nov 2005
Posts: 664

Posted: 09 December 2005, 18:46 PM    Post subject: serial port, tasks, GPS , and average processing power Reply with quote

THis is the continuation of a thread under a new more specific heading.

pdubinsky wrote:
The real problems started when I started converting from knots to mph and meters to feet. Occasionally, I'd just toss the whole queue out to catch up.


Hmmm.... I never noticed that I had that problem, even when I did some significant conversion of data. I wonder how efficient your parsing algorithm is....

At 4800 baud, you get about 480 bytes per second(0.002 seconds per byte). At about 5 bytes of ASCII per numerical value, you have 0.01 seconds to process a byte to keep up at max GPS NMEA speed. (One option is to slow the baud rate!!!!!)

I use a simple parsing technique.

I look for the "$" sign which start an NMEA sentence.

I then read 5 ascii bytes and compare that string to the NMEA identifiers I care about.

If it is a sentence type that I do not want, I just read stuff off the queue until the next "$".

If it IS a sentence I care about, usually I do not use ALL the data in the sentence.

I then skip data by looking for commas. When I have skipped over far enough, I read the requisite number of bytes, translate as needed, and keep going.


Your problem is that while you probably have an AVERAGE processing speed that is adequate to do all the work, all the data arrives in a huge dump all at once.

So, Is there any way to spread out the processing. I.E. Disconnect the translation/decoding of the data from the parsing of the stuff in the receive queue?

Maybe yes.

Howabout 2 tasks. The first task is to parse the NMEA sentences into the requesite text strings. When looking for a "$" sentence inititiator check for an empty queue, if it is then unlock the task and issue a Sleep(0). This will allow a task switch. When you have text coming in, then lock the task, to prevent buffer overflow.

The other task does the decoding/translation. You have the global text strings filled by the first task which may have new data to work on. You start at thefirst string and check each string, If it has fresh data, then do the decoding/translation and put the numerical values in a different global variable. Once you have used the string, then clear it so you will not re-do the same data. Maybe you could even test to see if it is the same thing you did last time. Often is, and maybe you can save some time there. Issue a Sleep(0) after testing/doing each individual string to allow a task switch. When you reach the last string, go back to the top and start over.

In this manner you have 2 independent processes, and the "receive" process (hopefully) is dominant by being able to lock itself, and it only allows switching to occur (within the limits of actually being able to control such on the ZX) when it is idle.


What do you think? I wonder how this will work with the limited stack space on the ZX. It probably would NOT work on the BX-24, but the additional spack space on the ZX may make this feasible.

Also, I wrote a few special-purpose ASCII-to-Numerical transation routines to try to improve speed over the general purpose routines, and reduce stack usage. Some/most of my attempts were WORSE than the built-in stuff.
Back to top
pdubinsky



Joined: 25 Nov 2005
Posts: 66
Location: South Carolina

Posted: 09 December 2005, 20:01 PM    Post subject: Re: serial port, tasks, GPS , and average processing power Reply with quote

spamiam wrote:

...

Howabout 2 tasks. The first task is to parse the NMEA sentences into the requesite text strings. When looking for a "$" sentence inititiator check for an empty queue, if it is then unlock the task and issue a Sleep(0). This will allow a task switch. When you have text coming in, then lock the task, to prevent buffer overflow.



I'm using comm 1 for the rcvr and I think that since the comm 1 uart service is basically a hardware interrupt driven task, I think that what you're sauggesting might already be happening. But if there's not enough time to actually ever process the data, then there is going to be a buffer overrun and the resulting chaos. What we haven't taken into account is my lousy coding which could be the culprit <gg>.

Quote:

...

What do you think? I wonder how this will work with the limited stack space on the ZX. It probably would NOT work on the BX-24, but the additional spack space on the ZX may make this feasible.


I think that the ZX-24 is a huge improvement over the BX-24 if for no other reason than the amount of RAM available. I NEVER thought I'd ever write code that created a 100 byte receive queue! That solved a lot of problems right there.

A spin off benefit of gaining control over the GPS receiver is (as I mentioned in the other thread) changing the baudrate to 38400. The amount time spent processing the uart at 38400 is 1/8th that at 4800. For a 75 byte GPGGA sentence, it isn't much but faster is better..


Quote:


Also, I wrote a few special-purpose ASCII-to-Numerical transation routines to try to improve speed over the general purpose routines, and reduce stack usage. Some/most of my attempts were WORSE than the built-in stuff.


I've tried that but NONE of my routines were any improvement.

Paul
Back to top
spamiam



Joined: 13 Nov 2005
Posts: 664

Posted: 09 December 2005, 20:57 PM    Post subject: Reply with quote

Well, Com1 is the hardware port, and is interrupt driven, but the interrupt ONLY puts the data in the queue. The task I am suggesting is more complex than that. It will read the queue and break out the data into its component ASCII parts, and make it ready to be translated/processed by a separate task.

The second task running concurrently does the translation/processing.

As far as Com1 baud rate. Running at a high baud rate is fine but your problem is that you can not process the data as fast as it comes in. Making it come in faster aggravates the problem and requires a larger queue because the overflow is worse. If you are old enough, you may have seen reruns of the old Lucy show where she is working on a candy assembly line. It runs faster and faster and she can not keep up. All sorts of highjinks ensue. That is what is happening to you.

By slowing down the baud rate so that you JUST finish reading and processing the last piece of data as the nest set begins will give you the most time to perform all your operations.

In the dual task model I suggest, you may be able to keep up pretty well with the 38000 baud data flow. By dumping it in FAST, it will allow the greatest separation (i.e. minimal task activity overlap) between the tasks.

I am not surprised that you have not improved much on the ZX general support routines. I think they are more thoroughly optimized than the stuff on the BX. It almost seems as if the BX compiler development was abandoned in mid-stream, and it is a work-in progress, without any more progress. I did check on the stack requirements for my special data conversion modules, but maybe I did it on the BX where we have a couple of programs to measure that stuff. I am pretty sure I also used some of the reporting functions of ZBasic to compute the likely stack demands.

So, in summary, I think that 38000 baud may stress your system worse, rather than less. At the very least it requires that you have as large a receive queue as the data you will receive over the full 1 second since there is no way you will be processing as it comes in.

At a lower baud rate, you might need a smaller queue because you can come closer to keeping up!

I still think that the 2 task technique is interesting, but maybe needlessly complex.

-Tony
Back to top
pdubinsky



Joined: 25 Nov 2005
Posts: 66
Location: South Carolina

Posted: 10 December 2005, 1:16 AM    Post subject: Reply with quote

spamiam wrote:
Well, Com1 is the hardware port, and is interrupt driven, but the interrupt ONLY puts the data in the queue. The task I am suggesting is more complex than that.

...



Right. I forgot about that. As you point out there's serious complexity involved which might be unavoidable if you're using the BX-24. The speed of the ZX-24 seems to obviate the need for that much compexity. It is something to remember in the future, though.

Quote:


...

As far as Com1 baud rate. Running at a high baud rate is fine but your problem is that you can not process the data as fast as it comes in. Making it come in faster aggravates the problem and requires a larger queue because the overflow is worse. If you are old enough, you may have seen reruns of the old Lucy show where she is working on a candy assembly line. It runs faster and faster and she can not keep up. All sorts of highjinks ensue. That is what is happening to you.

...



Not with this rcvr. The msgs show up once per second regardless of the baudrate. They can even be slowed to once per 255 seconds. But since I'm polling the GPS rcvr there are no unsolicited msgs showing up. No data shows up until the program asks for it.

Paul
Back to top
cloxwerx



Joined: 01 Dec 2005
Posts: 37
Location: Tucson, Arizona

Posted: 10 December 2005, 2:49 AM    Post subject: Reply with quote

Paul,

If you would like to see some code I wrote for the BX-24 that has been cranking for over a year reading GPS sentences at 4800 baud (COM3) I'd be glad to post or email it for you. It's part of an amateur radio APRS (Automatic Position Reporting System) project I built.

I'm going to enhance the program using ZBasic since there is more RAM and 2x speed available.

Dennis
W7KMV
Tucson
Back to top
cloxwerx



Joined: 01 Dec 2005
Posts: 37
Location: Tucson, Arizona

Posted: 10 December 2005, 3:36 AM    Post subject: Reply with quote

Paul,

Well, I forgot I sent you a link to the code on a previous message. Hope you get a solution running.

Dennis
Back to top
pdubinsky



Joined: 25 Nov 2005
Posts: 66
Location: South Carolina

Posted: 10 December 2005, 12:54 PM    Post subject: Reply with quote

Yes, Dennis, you did and it was a great help getting started. I have worked with streaming serial devices before and your code and my code were functionally equivalent though your's was a lot cleaner and better organized than mine. (I'm a "get working - we'll clean it up later" type coder.)

The real help I got from your code was how to work with the SiRF commands. Initially, I had trouble finding any useable documentation for the SiRF binary protocol so I was stuck on determining checksums (it was a while before I found that NMEA checksums and SiRF checksums were calculated differently), etc. When I finally got the necessary docs and understood them, the project moved right along.

The version of my program as of last night first initializes the GPS by turning off all the NMEA msgs. (This receiver sends all six standard NMEA sentences every second continuously at 4800 baud as the startup default.) Then I change the GPS serial baudrate to 38,400, change the comm1 baudrate to 38,400, and start a do loop polling for the GPS for the GGA sentence (to get altitude) and then polling for the RMC sentence (for Lat, Long, Speed, Track, Time and Date.) By polling and using the increased RAM of the ZX-24 (boy, do I like the increased RAM!), I don't have any possiblity of buffer overruns.

I'll drop you a zip file of the code when I've got it tidied up. I'm also planning on posting a code snippet showing how to use polling rather than streaming with the GPS receiver. I can't be the only person who has had this problem. And, using polling, the code should work on a BX-24 after the RAM usage is pared back down to fit the limited RAM of the BX-24.

Thanks, again, for the help,
Paul


cloxwerx wrote:
Paul,

Well, I forgot I sent you a link to the code on a previous message. Hope you get a solution running.

Dennis
Back to top
cloxwerx



Joined: 01 Dec 2005
Posts: 37
Location: Tucson, Arizona

Posted: 11 December 2005, 6:15 AM    Post subject: Reply with quote

Paul,

Thanks for the kind words about the code I sent to you. I'm glad it was of some use. I had used the MicroBlox test program that was available to exercise the GPS device I had and it generated all the hex commands that I copied out. It made it a lot simpler than calculating the checksums and parameters. I also didn't have to generate SIRF commands on the fly, so there was no need for calculation once I figured out what to do.

Good luck on your project.

Dennis
Back to top
Display posts from previous:   
Post new topic   Reply to topic    Forum Index -> ZBasic Language Time synchro. with the server - Timezone/DST with your computer
Page 1 of 1

 


All content Copyright © 2005-2012 Elba Corp. All Rights Reserved.
Opinions expressed in posts are those of the author and not necessarily those of Elba Corp.
Powered by phpBB © 2001, 2005 phpBB Group