| Author |
Message |
edkelly
Joined: 20 Jun 2008
|
|
Posted: 08 September 2008, 3:43 AM Post subject: ZX-24p Locking up or Resetting |
|
|
Any one have ideas why my ZX-24p is resetting or sometimes locking up? Without going into a lot of detail the program only acts up when I leave it running for several hours. Under testing it works fine but when I set it to run for a long period I come back only to find it has locked up or reset.
I'm running four tasks and each have a stack size set to about 150 more than the compiler asks for. If more information is needed I'll be happy to answer any questions that would help with the problem.
If any out there have thoughts or know of some posts that will help I'd greatly appreciate it.
Thanks so much...
|
|
| Back to top |
|
 |
dkinzer
Site Admin
Joined: 03 Sep 2005
Location: Portland, OR
|
|
Posted: 08 September 2008, 15:12 PM Post subject: Re: ZX-24p Locking up or Resetting |
|
|
| edkelly wrote: | | If any out there have thoughts or know of some posts that will help I'd greatly appreciate it. |
This type of problem is particularly vexing and it is made more difficult due to the fact that the actual cause of the problem may be separated in time from when symptoms begin to appear.
Have you checked your code to ensure that you are not invoking each task more than once (assuming that they do not terminate)?
You may be able to get some information about the cause by removing half of the functionality and then running it to see if the problem persists. If it works OK, then add back the parts removed, remove the parts just tested and repeat. If this process identifies a subset of the code that exhibits the problem, repeat the process on that subset. The idea is to reduce the code to the minimum that exhibits the problem.
If you could post the .map file produced by the compiler, that may provide some insight. The sections that I would like to see are the ones with these headings:
| Code: | --- Statically Allocated RAM Variables
--- Task Invocations and Stack Information |
|
|
| Back to top |
|
|
Don_Kirby
Joined: 15 Oct 2006
|
|
Posted: 09 September 2008, 0:17 AM Post subject: |
|
|
I've had many the same problem. As Don points out, the debugging method is typically to build the code a peice at a time until the problem appears, thus narrowing the problem down to one particular code snippet.
In my own experience, I've found it useful to output some diagnostic information via Debug.Print or Console.Write statements placed throughout the code. Even a simple "Hello, I'm working" can turn up non-functioning code.
It's also been my experience, that one should make sure there's gas in the car before one opens the toolbox. Check your power supply, and add a decoupling cap if there isn't one already. If you're experience level is beyond this, I apologize, but I've personally spent many days tracking down what I thought was a software glitch...
If you typically leave your device connected to your computer all the time (I do), perhaps the reset is coming from the serial port. Windows will toggle DTR as it boots, causing hangs and resets.
-Don Kirby
|
|
| Back to top |
|
|
dkinzer
Site Admin
Joined: 03 Sep 2005
Location: Portland, OR
|
|
Posted: 09 September 2008, 0:48 AM Post subject: |
|
|
| Don_Kirby wrote: | | I've found it useful to output some diagnostic information via Debug.Print or Console.Write statements placed throughout the code. |
One thing that is important to keep in mind when doing this is that, depending on what you output, you might affect the heap in ways that either mask the problem or make it worse. Outputting a string constant does not affect the heap but if you append strings or perform a value-to-string conversion, the heap will be used.
The function below may be useful for outputting a value without affecting the heap. Note that it could have been implemented as a subroutine; it was implemented as a function for the sole reason that it can be used in a semicolon separated Debug.Print list. The same idea can be used to implement similar routines for Byte, Integer and Long values.
| Code: | '
' This function outputs the decimal value of the supplied parameter to Com1.
' It is defined as a function returning a String only so that it can
' be used in a Debug.Print list.
'
Function DisplayUVal(ByVal val as UnsignedInteger) as String
Dim div as UnsignedInteger = 10000
Dim oq() as Byte Based Register.TxQueue
Dim digCnt as Byte = 0
Do While (div > 0)
If (val >= div) Or CBool(digCnt) Or (div = 1) Then
Call PutQueueByte(oq, LoByte(val \ div) + Asc("0"))
val = val Mod div
digCnt = digCnt + 1
End If
div = div \ 10
Loop
DisplayUval = ""
End Function
|
|
|
| Back to top |
|
|
edkelly
Joined: 20 Jun 2008
|
|
Posted: 09 September 2008, 11:00 AM Post subject: |
|
|
Thanks for all the input guys!
I guess it's only far to spend a moment explaining a bit of programs design. At the time of my last post I didn't have the time to do so. I'll try to stay brief.
I've built a small battery charger that runs on it's own independent of the ZX. The goal is to run a 12v battery through a series of charge and discharge cycles and monitor the effect of the charger on the battery. The charge and discharge times are stored in an array for to be viewed an entered into a spreadsheet at a latter time. The ZX's job is to monitor the batteries voltage, take it off charge at the set voltage level, let it rest for an hour, save the voltage and then place the battery on a small load that discharges it down to a set voltage, let it rest for an hour, save the discharged voltage and then start a new cycle.
Outside of main() there are four tasks that are called (main() simply calls the tasks and then stays in an empty Do Loop).
1. There is a Screen task that is by far the largest part of the program. It sets up a 4x20 LCD display and then shows any number of screen layouts pertaining to what mode the program is running in at the time. There are a number of vars that are constantly being updated and displayed.
2. There is a Buttons task which allows user interaction with the various screen information and to command the ZX. This task is the next largest part of the program and monitors a "Screen" var that so it stays in sync with the screen task.
3. There is that BattTest task which is only called when a battery load test is started. This is actually the work hours but is relatively small. It mainly is constantly (every 15 secs.) checking the batteries voltage and switching it from the three different stats of charge, rest, discharge and rest again. It also save the data in an array.
4. There is also a very small Blink task that aids in knowing if the ZX has stopped working all together. It blinks different patterns so I can quickly see what mode it is in with out using the screen which could be disconnected at any time and used in a different project.
The resetting or freezes seem to happen mostly during the first charge cycle within an hour or two.
As for shutting down different tasks that is a good idea I had not thought of, thanks! I'm not sure at this point what I can shut down and still have the program working. I will not be able to work with this project until the weekend.
As for the map file I've sent it along.
Thanks again ....
| Description: |
|
Download |
| Filename: |
Charger Map File.zip |
| Filesize: |
4.39 KB |
| Downloaded: |
285 Time(s) |
|
|
| Back to top |
|
|
Don_Kirby
Joined: 15 Oct 2006
|
|
Posted: 09 September 2008, 23:14 PM Post subject: |
|
|
| dkinzer wrote: | | [...]depending on what you output, you might affect the heap in ways that either mask the problem or make it worse. |
Excellent point. Typically, when I do this type of debugging, I just use string constants to see the program flow. Still, I'll stow away your function for future use, as there is no doubt I'll find the need for it at some point.
| Quote: | Outside of main() there are four tasks that are called (main() simply calls the tasks and then stays in an empty Do Loop).
|
I would suggest running just the Screen and Button tasks for a few hours without the Blink or BattTest tasks enabled. I'm putting my money on the BattTest task being the culprit.
When the application hangs, has the processor actually stopped running, or has the Screen task or Button task stopped responding?
When the application resets, how do you know? Perhaps the variable that indicates the reset is at fault (or being overwritten accidentally).
You do realize that we'll just keep spewing ideas until you tell us you've got it working properly...
-Don Kirby
|
|
| Back to top |
|
|
edkelly
Joined: 20 Jun 2008
|
|
Posted: 11 September 2008, 1:58 AM Post subject: More info on the ZX resetting or hanging |
|
|
| Quote: | | You do realize that we'll just keep spewing ideas until you tell us you've got it working properly... |
Yes it seems that is the case and let me just say how appreciative I am that you guys are willing to stick and hang with me! I usually only have the weekends to work on this project but your enthusiasm has gotten the best of me. So I'm trying to find some time in the evening hours. But know that if I don't respond write away it is only because I've not had a chance to try your ideas. And I intend to keep updating here as to my findings and progress.
| Quote: | | It's also been my experience, that one should make sure there's gas in the car before one opens the toolbox. |
Don, your comment earlier concerning the power source is well noted. I have thought of that before and did place a 470uF can across the supply. But after the problem persisted I removed it.
A word about me... My wife and I live off the grid on a 48v PV system that I put together three years ago. I'm using a small Solar Panel array that is rated at 800w (no way would it ever produce that much). With wood heat, a propane refrigerator and collecting rainwater off the roof and pumping it to the top of a 90' hill for pressure we live comfortably on an average of 3k Hrs a day. We've not experienced a brown out or loss of power for three years. The inverter is by OutBack and is a true sine wave inverter, very clean power.
Back on topic...
Don... I actually tried shutting down the Screen and Buttons tasks the other day and the rest of the program ran without a hitch for about 5 hours before I shut it down. I started it up again when I got home from work and will let it run through the night (we had some good sun today). I'm not sure disabling the BattTest task would show much because if there is no battery test going on the Screen and Buttons tasks are not doing anything but waiting for something to happen. If I wake up in the morning and the system is still running I will plan to reinstate the Buttons task. It would be the next easiest one to bring back. The Screen task is the by far the largest and has the most going on during a test. A number of vars being updated every seconds and the like.
There was also some concern as to why there were more then four tasks called in the .map file I sent along. Sorry for the confusion. The Screen task is called in two different places. The first time is from Main() to obvious get the screen set up and running. I have written a routine that allows me to shut the screen down and end the Screen task so I can unplug the screen and using it in other projects during a lengthy battery test which could go on for weeks. During this time the Buttons task is waiting for a certain key to be pressed which when pressed would indicate the screen had been plugged back in. At that time the Screen task is called again using the same stack to reset it and get it up and running again. This feature has worked well and proven very handing in the past. But I've been leaving the screen connected for the last few weeks and this feature has not been used during my recent testing of this problem.
The BattTest may also be called from two different locations in the Buttons task. One call would be a full battery test with up to 20 charge and discharge cycles. The other would be only one charge cycle used to simply charge a battery. Neither of the calls could coincide and the tasks are both exited when they are done or no longer needed.
The DegurVars is a task I forgot about but it was added only this last weekend in an attempt to see what was happening when the program went out the window. It simply spits vars out to a computer when they change using Debug.Print. The problem has been going on for several weeks so I'm not targeting that small routine at this time.
One other question... I was initially displaying the Register that saves the fault type at the startup of the program so as to maybe see what caused the reset. I was expecting numbers 1 through 4 but often saw 5 and if I remember correctly 8?!
And Don... I know the system is reseting when I leave the system in the middle of a test displaying all the voltages times and other vars and return to see the test halted and the opening screen as if I had pressed the reset button my self. During a test there is a motor running and relays clicking to place the batter in the circuit momentarily so it's voltage can be read... It's easy to see the system has gone to a place at the beginning of everything as though it was just turned on and waiting for a command. There have been a few time though that the ZX has stopped blinking, the screen has strange characters on it and stopped updating, the buttons are unresponsive and the charger is left in what ever state it was in when the program went bye-bye.
I'll let you all know in a day or so what I find out.
Thanks again for all your help and please continue with any questions or ideas.
|
|
| Back to top |
|
|
edkelly
Joined: 20 Jun 2008
|
|
Posted: 12 September 2008, 0:04 AM Post subject: It still resets... |
|
|
Well after a few hours into the test the ZX reset again. This is the test with the Screen and Buttons task shut down. Back to the drawing board. To shut the BattTest task down all that is nessessary is not to start a test. But then nothing is happening and the ZX is in a waiting state. 
Not sure what I'm going to try next. Any ideas?
|
|
| Back to top |
|
|
dkinzer
Site Admin
Joined: 03 Sep 2005
Location: Portland, OR
|
|
Posted: 12 September 2008, 0:53 AM Post subject: Re: It still resets... |
|
|
| edkelly wrote: | | Not sure what I'm going to try next. Any ideas? |
Rather than eliminating the entire BattTest task, start cutting things out of it. For example, you might comment out the code that activates the relays and then dummy up data, if necessary, to make the other parts of the code work. Gradually eliminate parts of the task until the problem no longer appears.
You might also read the Register.ResetFlags value when the code begins running. If the bit for the brownout detector is set, then you know that something caused the power to sag resulting in a reset. If no bits are set, it is likely a stack issue.
|
|
| Back to top |
|
|
edkelly
Joined: 20 Jun 2008
|
|
Posted: 13 September 2008, 13:10 PM Post subject: It's Still Resetting |
|
|
Wouldn't it work to showing the values of System.TaskHeadRoom for each task and find out which taskstack is being over run? Also I'm trying to find where in the Language Lib. talks about the StackLimit directive. The following sentence references it but I cant find it anywhere. 
ZBasic System Library Reference Manual (p. 246)
"See the Option StackLimit directive and related directives for more information."
|
|
| Back to top |
|
|
dkinzer
Site Admin
Joined: 03 Sep 2005
Location: Portland, OR
|
|
Posted: 13 September 2008, 15:16 PM Post subject: Re: It's Still Resetting |
|
|
| edkelly wrote: | | "See the Option StackLimit directive and related directives for more information." |
That reference is incorrect. The current version of the manual (available soon) says:
"See the Option MainTaskStackSize, Option HeapLimit, and Option HeapSize directives for more information."
In your case, I would suggest using the setting below since the .map file indicates that your Main() task requires at least 25 bytes.
Option MainTaskStackSize 50
Although your application may be suffering from stack overrun, I think it is unlikely because you have created generous stacks. It is more likely caused by heap corruption (or other RAM corruption) or brownout reset. It may also be caused by a yet-undetected error in the VM code. That's why you've received a couple of suggestions to keep narrowing down the locus of the code that is causing the problem. If you can identify a line or two that appear to be causing the problem, it is much easier to determine what is happening. Even if you can only narrow it down to 50 lines, that's better than having to examine the entire application.
If you can produce a test case that exhibits the symptom and doesn't require any external hardware, you can send it to me and we can analyze it further.
|
|
| Back to top |
|
|
edkelly
Joined: 20 Jun 2008
|
|
Posted: 14 September 2008, 0:19 AM Post subject: Still more info for the naughty ZX |
|
|
Ok here is some more info on the resetting problem...
I've updated the firmware on the ZX to 5.1.
I reinstated all the tasks so the program was fully operational again. Then I started Debug.Printing all the "System.TaskHeadRoom(taskStack)" for the all the tasks running. I found that nun of the stacks came even close to being used up. The program ran longer then it has ever run before. About 10 hours. I was starting to think that the firmware update did the trick. Finally in the 2nd charge cycle the ZX-24p reset. This time I was showing the Register.ResetFlags at startup.
The system was reset by the WatchDog. The "Register.FaultType" was 1 indicating a stack fault condition. "Register.FaultData" showed Hex 09ab or Dec. 2475 telling the address of the task that was running when the stack fault was detected. "Register.FaultData2" showed Hex 0243 or Dec. 579 telling the address of the instruction that was executing at the time of the fault.
Can we use the "Charger.map" file (latest copy attached) to gain any information about what happened? I viewed the .map file looking for the addresses indicated but I want to hear your interpretation of all this before I jump to conclusions.
I was hopping that showing the "System.TaskHeadRoom(taskStack)" for all the stacks would tell me which if any were over running. But that didn't seem to help. I can still do as you asked and narrow down the code and try and duplicate the problem but it usually takes a long time for a reset to occur which makes testing almost impossible.
| Description: |
| Latest .map file for this program. |
|
Download |
| Filename: |
Charger.map.zip |
| Filesize: |
4.74 KB |
| Downloaded: |
272 Time(s) |
|
|
| Back to top |
|
|
twesthoff
Joined: 17 Mar 2006
Location: Virginia, USA
|
|
Posted: 15 September 2008, 15:02 PM Post subject: Still more info for the naughty ZX |
|
|
Ed,
Did you get the private e-mail I sent you about the batteries?
Tom
|
|
| Back to top |
|
|
|
