Planning a DIY Data Acquisition/Engine Controller

[BruceM]

Yes, the I2C would be a good solution for co-processors, assuming the Arduino code supports both master and slave functions. Study, study!

In order to suggest an appropriate serial link design, I'd have to know what your plans for it are- distances and amount of data, and rate (data block transfer in Hz), and who (which processor) needs to talk to who.  You are likely getting carried away, a ring puts a burden of passing along data on every node, and you would more likely have a star or less. Your system is very small, not very complex from what you've described. It's a no-brainer, once you decide what it is you're doing.

I can't offer help with Java or PC GUI suggestions, I have zero experience with them. But I think there might be some guys here who might be able to help WJ in the right direction??? Anyone willing to share their experience and do a little teaching?  I'd love to learn about it. GUI's came after my time, and I never worked professionally with PCs. 

Best Wishes,
Bruce

[Crumpite]

Yes, the I2C would be a good solution for co-processors, assuming the Arduino code supports both master and slave functions. Study, study!

In order to suggest an appropriate serial link design, I'd have to know what your plans for it are- distances and amount of data, and rate (data block transfer in Hz), and who (which processor) needs to talk to who.  You are likely getting carried away, a ring puts a burden of passing along data on every node, and you would more likely have a star or less. Your system is very small, not very complex from what you've described. It's a no-brainer, once you decide what it is you're doing.

I can't offer help with Java or PC GUI suggestions, I have zero experience with them. But I think there might be some guys here who might be able to help WJ in the right direction??? Anyone willing to share their experience and do a little teaching?  I'd love to learn about it. GUI's came after my time, and I never worked professionally with PCs. 

Best Wishes,
Bruce

Folks,
I agree with Bruce (again!) - any type of topology that has multiple receivers/transmitters on the same buss can get *very*complicated in a hurry. (look at Ethernet !)
What you save in wire is more than made up in loss of hair and the cost of faster/more powerful micros to handle all of the overhead.
It would likely eliminate the possibility of using Picaxe's except for the most powerful ones.

I've done a fair bit of work on GUI's but always with a proprietary language, which doesn't help here any except in the most general terms.
Java seems to be the popular language, but I don't know a thing about it either. (another language to learn - arrgh...)
However I hear that it's based on the C language, so it should be similar to the Arduno language...
There do appear to be lots of examples of Arduino/PC communications out there, which will help a lot.

I've got my Arduino Duemilanove and a few other tidbits on the way, so I should be up to speed in a year or so. 

dubbleUJay, I already downloaded the manual you pointed me to and gone through it - it's a nice little starting place.

Well, back to the salt mines...
Daryl

[dubbleUJay]

Guys, here's a couple of thoughts as I see this project further down the line:

My DataLogger/Controller we're busy with is just one of the "nodes" of a modular system.
If we use the same micro, hardware, programming language and interconnecting structure, anyone can design a module and put it on the overall system.
Most of the nodes will probably only have a few inputs for data logging and maybe 1&2 outputs to switch something on/off.
I can name a few modules just of the top of my head:
Solar array data logging
Wind turbine logger
Water pump control
Simple irrigation system
exec. exec.

Lets say that the furthest module would be 100m away from the main controller/PC, after that I'll go with a wireless link.

Now for a biggish module like the Engine logger/controller a full blown Arduino make sense, but for most of the other a bare-bones-board would do. (A Kit cost about $12)
As long as they all connect to a central hub/PC in the same manner or talk the same language for that matter.
We have the option of incorporating a SD Memory Card for storage on each module as well, but I don't know how viable that would be. Maybe a single Arduino hub (with storage) were to all the modules connect to as a front-end would do (Daryl, I think this it what you meant a few posts back)

Most of this data can be sent to the central controller anytime within reason if "decision making" is done within each module like Daryl suggested.

Now what type/speed of comms we need to accomplish this, I don't know
We have to consider that a PC would not be on the whole time as well.
I'm probably missing a couple of point here as well.  

I've included a few pictures and info on some available boards like bare-bone & Serial interfaces just to get an idea of things, in the attached PDF.

[BruceM]

A couple thoughts-  First, thanks Daryl, you are much appreciated!

WJ, sure, a distant data collection situation cries out for a processor with a serial link.  At 100M I would recommend plastic optical fiber (Industrial Fiber Optics stuff is cheap and works great), if you want lighting proof, else the opto-isolated current loop on impedance matched twisted pair.

I'm using a I2C flash memory chip in my battery bank controller for battery charge cycle data logging. A little 8 pin dip with 1 Mbit of space for $3. It's just dip socketed so I can pull a tab and swap it out for data retrieval (Picaxe chip as reader using serial to USB link to PC), or use the slow, 600 baud opto-isolated current loop serial data link down to my shop located computer for just pulling the most recent time stamped records.

I'll bet there's some nice Arduino board with a memory stick or flash memory card of some kind, if you've got the power and $.

I'm not sold on the boards, though it is a tough choice given the Arduino bargains- I always need some analog and digital "glue" chips, so if I'm making a board, I'd rather just put the processor on it and do one board with less connectors, and I get just what I want, instead of tacking on a second board to what someone else thought I'd want.

[dubbleUJay]

Bruce, I'm quite happy with twisted pair and I'll use that in most of the applications, but what I dont know is how would one switch between the different modules from the PC side of things?
I've only dealt with 1to1 comms.
The multi-systems I worked with i.e. PTZ control and addressable fire alarm equipment all had dedicated equipment controlling them, so I'm kind of lost on this one.
I presume that each module will have some type of address to be able to communicate with them individually?
Or maybe I'm missing the point here guys

[Crumpite]

Folks,

DubbleUJay,
I'm envisioning this type of network:
First you have a Master. It's the first board you install in the system.
It has some sort of connection to your PC, whatever flavor you'd like.
It's only task in a properly designed system (which we may take liberties with) is to collect and store data and handle all of the communications with the PC and the Slaves.
It also has the higher level code for controlling the Slaves. (see below...)
The Master will need a serial com port for each slave and some kind of com link to the PC.
This may limit it's usefulness as a data collection device. (in which case we just add another slave node to do the data collection)

The PC has two functions, first to download data from the Master and display and/or store it.
Second, it can send commands to the Master, anything from "dump me your data" to "send a shutdown command to the engine controller."

There can be Slave boards connected to the Master. They do only two things: they collect data and store it only long enough to make decisions on, and send to the Master.
They also do all of the actual control of the system - they have outputs that control things and the code to do it.

This system has many advantages: the Slaves can be simple, low cost units. All they need are inputs, outputs, the control code and the communications code.
If needed they can do a little data reduction and short term storage, but only enough to get by.
This means the program is pretty simple, and the communications code can be reused from Slave to Slave.
Also, the Slave should control only one device, like the engine, or battery management, or wind turbine.

The Master contains the higher level code that coordinates between the slaves.
If the battery management Slave says the battery bank is full, the Master sees that in the data and it's control code sends a shutdown signal to the engine Slave.
(or whatever)
It has the potentially expensive storage system, and communications link to the PC.

Each subsystem can work without the others there.
If a mouse nibbles through your wire out to the engine shed, the engine continues to run and is still protected by it's separate controller.
You just loose the data that was being sent, and the ability to control that subsystem. So you walk out there and hit the shutdown button manually.
(and find the break in the cable !)
Of course you knew about the cable break because the Master noticed that there was no communications and sounded some sort of alarm !

Bruce,
I know what you're saying about the boards. It's much more expensive to buy general purpose boards, plug them together and add your own interface.
But part of the idea here is to make this thing usable and less intimidating to fledgling CHP folks. 
However, with all of the schematics and Eagle files out there for the Adrundo system, it would be simple to make up our own version of "CHPundo" or whatever. 

Got to run,
Daryl

[dubbleUJay]

Now Daryl, WHY could I not put it like that into words

That's about the way I envisioned it!

Another nice thing about this way is if a guy just want an engine controller, he can just get/buy that hardware for the module, load the code into it and he has a stand-alone version, which, if we play our cards correctly, could still plug into the same port as the main controller would have to the PC and get control via it. (I think???)

I don't want to jump the gun, but we should be able to do the controller with an Arduino as well if the following might work?

1) SD card & USB sticks has already been done with the Arduino, although it uses a lot of code apparently, unless one uses a "smart" reader for them with real-time clock, UART/SPI interfaces exec.
(I've got to look into this more, but here is a link to one: http://www.sparkfun.com/commerce/product_info.php?products_id=8215 )

2) The 4067 that Bruce suggested is a 16ch Mux-/Demux which probably comes in a digital form somewhere? Without looking back to Bruce's optical-232 (2wire or 4wire ?) The code to control 2or4 of them to switch the wires between modules should not be difficult or huge. That's if one can switch 232 with them, I don't know if its fast/good enough?

I'm calling it a day!
WJ

[BruceM]

If you are going to expand the remote master-slave star extensively, to say 8 slave nodes, for fun or whatever, then you  can use cmos digital mux/demux chips (not analog switches) for the master to select the appropriate slave. One output, one input per slave for asynchronous serial data. (Two pair- or a single pair  if half duplex.)

If you have lots of processors that are in the same room (relatively short distance but you don't want to co-locate and use I2C bus), then busing the serial lines between them is a solution.  Depending on the data rates, this could even be half duplex, a single wire with pull ups, open collector drivers, data packet id headers (just a byte) to tell which slave's data it is) will simplify the hardware. Each slave on the bus does have to read every data packet to determine if it's "his", and data packet integrity must be checked, which is overhead you'd like to avoid unless it makes sense for the system overall.  If I2C is used with a chip that does in hardware, like the Picaxe X2's that overhead becomes relatively transparent to your processor. 

It's all largely a beer fart until you define the actual data, rates, and distances, at least as a preliminary estimate.  Planning expansion capability is good, but keeping things practical  and simple helps others who might like to use it, too.

Great, thoughtful comments on the design and Arduino hardware use, Daryl. Applying the Arduino stuff that's out there well is a good idea.  I'm biased towards greater simplicity and am more of a roll my own type, but appreciate the value of the use of the Arduino gear for an "open" project. 

[dubbleUJay]

If you are going to expand the remote master-slave star extensively, to say 8 slave nodes, for fun or whatever, then you  can use cmos digital mux/demux chips (not analog switches) for the master to select the appropriate slave. One output, one input per slave for asynchronous serial data. (Two pair- or a single pair  if half duplex.)

Thank you Bruce, I've got a feeling that this might be the way to go. I'm thinking if one are going to use a Master Module it might as well have at least 8 I/O comm ports for future expansion.

If you have lots of processors that are in the same room...............

IMHO this will probably be the exception to the rule, in my case anyway. The only place I can think of right now where I'll use 2 is if I split the Logger/Controller, but there will be other instances.

It's all largely a beer fart until you define the actual data, rates, and distances, at least as a preliminary estimate.  Planning expansion capability is good, but keeping things practical  and simple helps others who might like to use it, too.

You guys are much more clued-up on this than me, but maybe the Engine Logger/Controller would give a good indication of what to expect, at least I now know that it should do as much of the calculations and decision making by itself.

I'm biased towards greater simplicity and am more of a roll my own type, ..............

You wouldn't be an EE if you weren't Bruce You probably saw that I would rather also build my own sensors & stuff, but I keep on thinking of the next guy! I would love for more people to maybe build this thing when its finish, everyone that has one of these would be able to quote some darn accurate figures I would think, while also using it for something useful like control???

This is wishful thinking, but there are even guys logging there data to a communal web-page, like power used, rainfall, wind-speed exec., the stuff one can do is mind-boggling, for me anyway!
Imagine Daryl phoning you and saying that he saw on the website that your wind generator's output is 3kW in 30km/h winds, how-u-do-that?
Probably not in my lifetime though!

[BruceM]

You're doing a great job, WJ. We'll Arduino the heck out of it.

When you get a chance, make a list of your full system (including noted dreams) functions, list of subsystems, and the rough distances between them.  We can then discuss the data and data rates.  Then Daryl and I can help you turn that into a preliminary spec and communications design that will be ready to do everything you might need, simply and reliably.

[dubbleUJay]

Thanks Bruce, Daryl has thrown the stone in the pond and we might as well do it all the way! Within reason obviously. I believe this is where it would have lead to eventually anyway.

I also believe Daryl is busy with some type of flow chart as far as I know? Daryl
We can take it from there and add/agree about things?

Hope you recover soon! 
dubbleUJay
PS- I've got a hidden agenda, "Arduino" the sh!t out of National Instruments over-priced student edition hardware/software

[dubbleUJay]

.................. CHP folks.  .................the Adrundo system, it would be simple to make up our own version of "CHPundo" or whatever. 
Daryl

I'm getting carried away again, but maybe "McGuino"

Not as in "McCreary" or "MacGuyver", but as in "Micro Co-Gen"! 

dubbleUJay (just foolin' around )

[Crumpite]

Folks,

Well right now I'm studying the heck out of ways to do the whole PC/Master control scheme... Argh... My eyeballs hurt...

There is *so* many solutions to the problem it boggles the mind. None of them so far seem to be what we're looking for.

I've got another book to go through tonight, perhaps it will have a few clues as to how to do it.
So far I'm leaning towards some form of visual basic, but that will likely change 17 times in the next several days.
I wish I had more time to put in on this research - I'm commented to helping some friends out with an instrument problem, and it's taking a lot longer than I thought it would.

/rant mode on/
There are so many competing languages, non of which are aimed at what we want to do. So much web development "junk"...
Well, I consider it junk at least. I can do without all of this flash and dazzle on the screens these days.
What ever happened to "form follows function ?"
/rant mode off/

Well, back to the salt mines, my PDF is just about ready to be downloaded into my ebook reader for my bedtime enjoyment !
Daryl

[dubbleUJay]

Daryl, on the master hardware side, Bruce has given me a sort of thumbs-up on using my multiplexer idea to switch between the various slaves. He made a suggestion on IC's to use, although he said he's still looking for one that might work better.
I'll draw up something "alpha" 2day to look at.
Do you have any cct from him about his opto-RS232 links? I need to see how many "wires" needs to be switched for each slave?

[mobile_bob]

perhaps i am taking a simplistic approach to automation, basically because of my experience in
the mid/heavy duty trucks and also some of the plc's used in industrial climates.

personally i see little to no need for a ton of data logging, and what little i see the need for can be done
with simple setting of error codes, which can be done with a few line driven led's. give me three lines and i can
give you 9 possible error codes add another line and you can have 16 and so on.  if i need more i suppose i could use
a multiplexer/demultiplexer to come up with tons of possible codes.

when you look at a computer controlled automotive application the controls that are used are numerous, everything
from manifold pressure, fuel pressure, oil, water, on and on and on. none of them require a pc screen or human intervention
to operate just fine, and when human intervention is needed you can get into the puter with a 20 dollar code reader and a code
book to find out what the issue is.

now this is not to say that having it all on a puter screen would not be cool, actually it would be very cool, however
it adds another layer of complexity that we will find will come back to haunt you big time.

as an example, i work on cnc controlled component saws in the truss industry, you got the plc's doing the control, and you have
a remote pc and screen to setup the job, calibrate the machine, run the job and display it all.  and you have tons of communication
cables all of which become very finicky over time, creating issues that drive you out of your mind trying to sort out.

bear in mind these are all hardened industrial computers, and almost every one i am aware of has numerous major failures and problems before the one year warrantee is out, and it does not improve with age after warrantee.

in my view, plc control has proven to be very reliable and anything we build that emulates their design is a step in the right direction.

now most plc's use ladder logic (relay logic) however there is no reason that other languages could not be used for our purposes.
its not the programming that fails down the road, rather the hardware and implimentation of that hardware.

successful designs seem to have the master and its slaves mounted in a common chassis, this allows for a high quality buss
without lengthy cables and connectors to degrade from various outside problems.

basically what i am saying is in my opinion any successful system should not integrate both data aquisition/logging and automation
they should be as seperate as is practical, the automation system should be rock solid, simple, redundant where possible, and fail to safe
mode, with an operator override capability or manual control system overlay.

the data logging or aquisition system can then be any flavor you like, and can be as ruggedized as one feels necessary or can afford
weighing out the various factors and realizing that probably after a few weeks of oogling the screen, the thing will likely never be looked at again, at least with any regularity, so its need to be nasa quality is probably not necessary.

fault codes should be integrated into the automation control system, only because it would be easier, and because a fault is something
we will certainly always want to know about, even after the new wears off.

my concern is otherwise we end up with an integrated system that is the "jack of all trades and master of none" and likely will be finicky
and suffer all sorts of breakdowns and other anomolies that will be very hard to diagnose.

a couple years back i took on the reverse engineering of the automation direct dl305 plc system, it took me about a week or two to reverse engineer one of the 16pt output modules, another week or so to reverse engineer the backplane, and more time for input modules and i decided to forego the processor module because i knew i wanted to replace it with a micro processor that i could program
in pbasic.

yes i was warned and ridiculed by all the plc guys, got blasted by the folks over at automation direct for hacking their product, but
i learned a ton about proper design and how simple it can be if we use a wide buss rather than getting cute with I2C or other communications protocols,,, we don't need those communication protocols "if" we keep things close, such as in the same chassis
and use whatever extensions architecture to connect to the outside world.

we use 5 volt logic, plc's use other voltages that are much higher and there is no reason we can't use any voltage logic we want
to get from a controlled object and our controller.

the bottom line i decided to learn how things are controlled by the plc guys, also learn how the microcontroller guys do their thing
and then blend the two technologies with available hardware.

its hard for me to want to have pc boards made, source all the discrete components, learn more languages to program different
processors, and basically have to reinvent every wheel on this project along with the spare wheel in the trunk.  when just about ever
bit of the hardware is available for a very low cost, one just has to think seriously what he wants to accomplish, lay out a flowchart, then
assemble the hardware to enable control, leaving programming to tie it all together.

but what do i know?

nothing more than how i plan on getting the job done!



bob g