|
|
| Author |
Message |
pdubinsky
Joined: 25 Nov 2005
Posts: 66
Location: South Carolina
|
|
Posted: 08 December 2005, 15:55 PM Post subject: GetQueue Anomaly |
|
|
I've been testing the following code to see if I can move data between queues quickly. With the following code, if I try to move date from InQueue to TestQueue using GetQueue(InQueue, TestQueue, 100, 1.0, timeOut) it appears that only 67 bytes arrive in TestQueue. However, if I move the data a byte at a time using:
call GetQueue(InQueue, k, SizeOf(k), 1.0, timeOut)
call PutQueue(TestQueue, k, SizeOf(k))
all 100 bytes arrive.
I thought at first that there might be a speed advantage to moving 100 bytes as a block but now I don't know.
Any ideas?
Thx,
Paul
| Code: |
Option Explicit
Public InQueue(1 to 109) as byte
Public TestQueue(1 to 109) as byte
Public Sub Main()
dim BigQueue as boolean, timeOut as boolean
dim j as byte, k as byte, m as byte
dim t1 as single
call OpenQueue(InQueue, SizeOf(InQueue))
call OpenQueue(TestQueue, SizeOf(TestQueue))
BigQueue = false
'BigQueue = true
t1 = timer
for j = 1 to 100
k = &h40 + j
call PutQueue(InQueue, k, SizeOf(k))
next
debug.print "In Queue Count: " & CStr(GetQueueCount(InQueue))
debug.print "In Queue Buffer Size: " & CStr(GetQueueBufferSize(InQueue))
if (BigQueue = true) then
Call GetQueue(InQueue, TestQueue, 100, 1.0, timeOut)
else
for j = 1 to 100
call GetQueue(InQueue, k, SizeOf(k), 1.0, timeOut)
if (timeOut = true) then
debug.print "Failed"
exit for
end if
call PutQueue(TestQueue, k, SizeOf(k))
next
end if
debug.print CStr(timer - t1)
if (timeOut = true) then
debug.print "Failed"
else
debug.print "Test Queue Count: " & CStr(GetQueueCount(TestQueue))
debug.print "Test Queue Buffer Size: " & CStr(GetQueueBufferSize(TestQueue))
end if
k = 0
do while StatusQueue(TestQueue)
k = k + 1
call GetQueue(TestQueue, m, 1)
debug.print Chr(m)
loop
debug.print CStr(k)
debug.print "In Queue Count: " & CStr(GetQueueCount(InQueue))
debug.print "In Queue Buffer Size: " & CStr(GetQueueBufferSize(InQueue))
end sub |
|
|
| Back to top |
|
 |
dkinzer Site Admin
Joined: 03 Sep 2005
Posts: 2499
Location: Portland, OR
|
|
Posted: 08 December 2005, 17:41 PM Post subject: |
|
|
| Code: | | GetQueue(InQueue, TestQueue, 100, 1.0, timeOut) |
This will most definitely not work as you hoped it would because GetQueue doesn't know or care that the second parameter that you provided is, in fact, another queue. The second parameter is simply a reference to a variable where the retrieved data will be written. As written, the call will obliterate the queue management data for TestQueue.
To do what you're thinking about would require a special routine that would accept two queue references. An interesting idea, that. |
|
| Back to top |
|
 |
dkinzer Site Admin
Joined: 03 Sep 2005
Posts: 2499
Location: Portland, OR
|
|
Posted: 08 December 2005, 18:41 PM Post subject: |
|
|
If you want to experiment with this concept, here is some test code that relies on internal implementation details:
| Code: | Private Const qDataSize as Byte = 7
Private Const qOverhead as Byte = 9
Public InQueue(1 to qDataSize + qOverhead) as Byte
Public TestQueue(1 to qDataSize + qOverhead + 1) as Byte
Private testQueueCapacity as UnsignedInteger Alias TestQueue(1) ' the queue data capacity
Private testQueueCount as Integer Alias TestQueue(3) ' number of data bytes in the queue
Private testQueueHead as UnsignedInteger Alias TestQueue(5) ' the insertion point index
Private testQueueTail as UnsignedInteger Alias TestQueue(7) ' the removal point index
Private testQueueFlags as Byte Alias TestQueue(9) ' queue management flags
Private testQueueData as Byte Alias TestQueue(11)
Public Sub Main()
Dim j as Byte, k as Byte, m as Byte
Dim cnt as Integer
Call OpenQueue(InQueue, SizeOf(InQueue))
Call dbgDspRamBytes(CInt(InQueue.DataAddress), SizeOf(InQueue))
For j = 1 to qDataSize
k = &H40 + j
Call PutQueue(InQueue, k, SizeOf(k))
Next j
' copy data to the test queue
cnt = GetQueueCount(InQueue)
Debug.Print CStr(cnt)
Call OpenQueue(TestQueue, SizeOf(TestQueue))
Call GetQueue(InQueue, testQueueData, cnt)
' fix up the test queue tracking data
testQueueCount = cnt
testQueueHead = CUInt(cnt)
' display the data in the test queue
Do While (GetQueueCount(TestQueue) <> 0)
Call GetQueue(TestQueue, m, 1)
Debug.Print Chr(m);
Loop
Debug.Print
End Sub
|
The management data for a queue consists of the following:
| Code: | offset 0: capacity - the size of the data area of the queue
offset 2: count - the number of data bytes currently in the queue
offset 4: head - the index into the data area of the last byte inserted
offset 6: tail - the index into the data area of the next byte to remove
offset 8: flags - information about queue properties |
Due to the way the the head/tail indices are managed internally, the first byte to be added to a queue after it is initialized resides in the second byte of the data area, i.e. the head index in incremented before insertion. The tail index is incremented after removal. In both cases, when the index exceeds the data area size it is wrapped back to zero. Also note that head = tail both when the queue is full and when it is empty.
Because of this implementation strategy, the receiving queue needs to be one byte longer than the source queue in order to provide a contiguous region for GetQueue to copy data to.
Note, particularly, that the receiving queue must be empty before GetQueue is called. Otherwise, it gets very messy because of the way that the head and tail indices wrap around. |
|
| Back to top |
|
 |
pdubinsky
Joined: 25 Nov 2005
Posts: 66
Location: South Carolina
|
|
Posted: 08 December 2005, 20:04 PM Post subject: |
|
|
This is an interesting idea because the point of this was to speed up
moving the data out of a receive queue that was receiving data from a
source that the program could not control - in this case a GPS
receiver. The GPS rcvr sends 2 NMEA sentences every second which at
4800 baud doesn't take very long but processing the first sentence
causes problems receiving the second sentence. The idea is to receive
both sentences into a large receive queue (aprrox 120 bytes), move
that data to a processing queue and continue receiving data into the
now empty receive queue while the data in the process queue is "processed
The standard approach would eat up a lot of time spinning in the
GetQueue loop and not help much. I tried BlockMove which seemed to
work. But if it did actually work without clobbering the system, it
didn't empty the receive queue which would still have to be emptied in a loop.
I'll try this. and see how it works.
Thx,
Paul
| dkinzer wrote: | If you want to experiment with this concept, here is some test code that relies on internal implementation details:
| Code: | Private Const qDataSize as Byte = 7
Private Const qOverhead as Byte = 9
|
|
|
|
| Back to top |
|
 |
dkinzer Site Admin
Joined: 03 Sep 2005
Posts: 2499
Location: Portland, OR
|
|
Posted: 08 December 2005, 20:24 PM Post subject: |
|
|
| The other approach that may yield the performance you're seeking is to do a GetQueue() to an ordinary byte array and process the data from there. |
|
| Back to top |
|
 |
spamiam
Joined: 13 Nov 2005
Posts: 664
|
|
Posted: 09 December 2005, 13:26 PM Post subject: |
|
|
| pdubinsky wrote: | The GPS rcvr sends 2 NMEA sentences every second which at
4800 baud doesn't take very long but processing the first sentence
causes problems receiving the second sentence. The idea is to receive
both sentences into a large receive queue (aprrox 120 bytes), move
that data to a processing queue and continue receiving data into the
now empty receive queue while the data in the process queue is "processed. |
Well, I have implemented a GPS interpreter. I have it hooked up to 2 different types of GPS receivers. I do not need 2 separate queues. I use 1 queue that is long enough for a full second of data.
These NMEA sentences have a lot of extraneous info, and much of it is able to be skipped. In fact, many entire sentences are able to be skipped, at least most of the time.
Even on the lowly BX-24 I could read lat, long, elev, vx,vy,vz, speed, as well as time with no trouble, and with no data loss.
Have you tried this task with just the one receive queue and lost data? How much additional processing of the data are you doing? I am just translating the text to numerical format, and displaying the data. I am pretty sure the parsing and translation from ascii to numbers is by far the most time consuming task unless you are doing trigonometric functions on the lat/long. If you are doing so, (e.g. Autonomous Vehicles), then I have found that linear math is fine when you have made a correction for the latitude "miles per degree".
-Tony |
|
| Back to top |
|
 |
pdubinsky
Joined: 25 Nov 2005
Posts: 66
Location: South Carolina
|
|
Posted: 09 December 2005, 17:46 PM Post subject: |
|
|
Tony,
The real problems started when I started converting from knots to mph and meters to feet. The increased speed of the ZX-24 and its additional ram made a big difference. But I was still spending a lot of time tossing out NMEA sentences that I didn't want. Occasionally, I'd just toss the whole queue out to catch up.
That all changed with a little research. Now I send the GPS a series of msgs that stop the GPS from automatically sending the default NMEA sentences. That gets the GPS to stop sending anything though it's awaiting instructions. Then I poll for the sentences that I want when I want.
I'm interested in the GGA sentence and the RMC sentence. When I poll for the GGA sentence, the GPS responds with the same data that it was originally spewing but now it only sends it once. I parse the sentence, do my conversions and display the data. Then I get the other sentence and do the same. Loop continuously. Even the BX-24 could keep up this way since the conversion time overhead doesn't matter.
Paul
| spamiam wrote: |
...
Even on the lowly BX-24 I could read lat, long, elev, vx,vy,vz, speed, as well as time with no trouble, and with no data loss.
Have you tried this task with just the one receive queue and lost data? How much additional processing of the data are you doing? I am just translating the text to numerical format, and displaying the data. I am pretty sure the parsing and translation from ascii to numbers is by far the most time consuming task unless you are doing trigonometric functions on the lat/long.
.....
-Tony |
|
|
| Back to top |
|
 |
spamiam
Joined: 13 Nov 2005
Posts: 664
|
|
Posted: 09 December 2005, 18:18 PM Post subject: |
|
|
| pdubinsky wrote: | | The real problems started when I started converting from knots to mph and meters to feet. Occasionally, I'd just toss the whole queue out to catch up. |
Hmmm.... I never noticed that I had that problem, even when I did some significant conversion of data.
At 4800 baud, you get about 480 bytes per second(0.002 seconds per byte). At about 5 bytes of ASCII per numerical value, you have 0.01 seconds to process a byte to keep up at max GPS NMEA speed.
I use a simple parsing technique.
I look for the "$" sign which starta a NMEA sentence. I will put it here in this forum section.
-Tony
I then read 5 ascii bytes and compare that string to the ones I care about.
If it is a sentence type that I do not want, I just read stuff off the queue until the next "$".
If it IS a sentence I care about, usually I do not use ALL the data in the sentence.
I then skip data by looking for commas. When I have skipped over far enough, I read the requisite number of bytes, translate as needed, and keep going.
I will make a new thread at this point because this could get a little interesting, and I don;t want to hide it in this thread.... |
|
| Back to top |
|
 |
pdubinsky
Joined: 25 Nov 2005
Posts: 66
Location: South Carolina
|
|
Posted: 09 December 2005, 19:30 PM Post subject: |
|
|
I do the same but I have two complicating factors.
1. The rcvr I'm using defaults to sending 6 different sentences per second and just dumping the excess is time consuming and
2. I have to send about 70 serial bytes to a display at 19200. Not a lot of data but slow and ...
By changing the parameters of the rcvr to just 2 sentences, polling them and changing the rcvr xmit baudrate to 38400, things have changed remarkably. At least the data stream is under my program's control - not the rcvr's control.
| spamiam wrote: | | pdubinsky wrote: | | The real problems started when I started converting from knots to mph and meters to feet. Occasionally, I'd just toss the whole queue out to catch up. |
Hmmm.... I never noticed that I had that problem, even when I did some significant conversion of data.
....
I look for the "$" sign which starta a NMEA sentence. I will put it here in this forum section.
-Tony
I then read 5 ascii bytes and compare that string to the ones I care about.
If it is a sentence type that I do not want, I just read stuff off the queue until the next "$".
....
I will make a new thread at this point because this could get a little interesting, and I don;t want to hide it in this thread.... |
Good idea on the separate thread... see ya there...
Paul |
|
| Back to top |
|
 |
|