Mar 192017

So, further progress on the charge controller.

Thinking about the problem … I realised that I really do not want to be testing for VBNVH when entering the CHARGE_CHECK state, as it’ll prematurely terminate the charge when the battery is being bulk-charged.

Better to wait until the charger decides to stop … which we’ll see due to the battery voltage ceasing to increase. We do want to check we’re not critically high however, so we can swap out VH for VCH.

Next, when we find that VBNVBL, meaning the battery is not charging, there we can check for VBNVH and stop charging at that point.

That change, worked pretty well, but it was still flapping between sources. A little state management helped this. If we declare another state variable, charger_warning, we can flip this to 1 upon first detecting that VBNVBL, then wait a little longer. If after a time-out this is still the case, then we can take action. Thus we define a new timer tCWARN, which delays acting on the not-charging case.

A bit of threshold tweaking, and things are behaving themselves. I’m using an el-cheapo 3-way camping fridge as the stand-in for the cluster. This is an Aldi special bought some years ago that draws about 5-6A… and when running on 12V power, features no thermostat.

We’re finding that its cooling capacity is no match for Brisbane’s early autumn weather anyway, so it’s pretty much would be a constant load even if the thermostat worked on 12V.

The battery we’re using is an old 105Ah AGM battery… which is one of two batteries from the caravan we have here. They were the original batteries, and this battery’s mate had failed when both were replaced. We’ve noted this battery getting warm whilst charging, so we think it might now be on the way out too.

What to replace it with? LiFePO₄ is AU$1000 for 100Ah, so too expensive. AGM is still the better bet. I can get 300Ah AGM batteries, but they weigh nearly as much as I do. I can just manage the 105Ah, so we’ll stick with those. I will need more than one long-term.

That brings the thorny issue of connecting them. I am not keen to hook batteries in parallel for various reasons. At least not permanently.

Now, the charger I’m using for mains is a 3-channel charger. I can make additional charge controllers (with the caveat that I need heatsinks for the MOSFETs…sigh!) and I can look for 3-channel solar chargers, or just get multiple chargers for the extra batteries. This can be done.

The load is the elephant in the room. I’d ideally like to manage it as a single load, although conceivably, I could put the switch and one storage node on one battery, a second storage node and a compute node on a second, and the final storage and compute nodes on the third. If the switch goes however, my cluster is toast.

I can put batteries in parallel, but this really does need to be done with care, using carefully matched batteries. So the better solution is to have a controller that chooses the battery with the highest voltage.

There might be an analogue means of implementing this, but a microcontroller is a single-chip solution. The ATTiny24As have up to 8 ADC/GPIO pins and three non-ADC GPIO pins. It’s what I’m already using for the charge controller, so is an easy choice.

The cluster will not tolerate a break-before-make switch-over. I thought about using a capacitor bank to keep the cluster alive during a brief (~1sec max) switch-over. Back-of-the-envelope calculations suggested I would need a 10F capacitor bank. I can get a 16V 470mF capacitor for AU$70 each… and would need 20 of these. Ouch!

A small battery is another option, maybe a 7Ah, but that has its own maintenance issues, and represents a single point of failure.

I can get Schottky diodes capable of 40A, they still present a 0.6V voltage drop. At 30A, that represents 16W! For comparison, a relay with a 225ohm coil resistance will draw ~60mA when the battery is at the maximum of 15V, representing a load of 1W.

Or I can use more MOSFETs like the ones I’m already using, which draw even less power; poor man’s solid-state relays. Latching relays also exist, but they can be rather expensive, more so than a solid-state relay.

I can probably get away with temporary parallel connections, so a make-before-break would let me switch sources. Or, I could place my switch across a Schottky, meaning I put up with that 16W load for a brief moment while I switch sources.

So more to think about, but we are getting close. I can defer this decision until I get a second battery, but I am getting close to the point where the cluster will be running full-time.

Feb 112017

So… in the last test, I tried setting up the nodes with the ATTiny24A power controller attempting to keep the battery between 11.8 and 13.8V.

This worked… moreover it worked without any smoke signals being emitted.

The trouble was that the voltage on the battery shot up far faster than I was anticipating. During a charge, as much as 15.5V is seen across the battery terminals, and the controller was doing exactly as programmed in this instance, it was shutting down power the moment it saw the high voltage set-point exceeded.

This took all of about 2 seconds. Adding a timeout helped, but it still cycled on-off-on-off over a period of 10 seconds or so. Waay too fast.

So I’m back to making the nodes more tolerant of high voltages.

The MIC29712s are able to tolerate up to 16V being applied with peaks at 20V, no problem there, and they can push 7.5A continuous, 15A peak. I also have them heatsinked, and the nodes so far chew a maximum of 3A.

I had set them up to regulate down to approximately 13.5V… using a series pair of 2.7kΩ and 560Ω resistors for R1, and a 330Ω for R2. Those values were chosen as I had them on hand… 5% tolerance ¼W carbon film resistors. Probably not the best choice… I wasn’t happy about having two in series, and in hindsight, I should have considered the possibility of value swing due to temperature.

Thinking over the problem over the last week or so… the problem seemed to lay in this set point: I was too close to the upper bound, and so the regulator was likely to overshoot it. I needed to knock it back a peg. Turns out, there were better options for my resistor selections without resorting to a trim pot.

Normally I stick to the E12 range, which I’m more likely to have laying around. The E12 series goes …2.7, 3.3, 3.9, 4.7, 5.6… so the closest I could get was by combining resistors. The E24 range includes values like 3.0 and 3.6.

Choosing R1=3.6kΩ and R2=390Ω gives Vout ~= 12.7V. Jaycar sell 1% tolerance packs of 8 resistors at 55c each. While I was there today, I also picked up some 10ohm 10W wire wound resistors… before unleashing this on an unsuspecting AU$1200 computer, I’d try it out with a dummy load made with four of these resistors in parallel… making a load that would consume about 5A for testing.

Using a variable voltage power supply, I found that the voltage could hit 12.7V but no higher… and was at worst .7V below the input. Good enough.

At 16V, the regulator would be dropping 3.3V, passing a worst case 3A current for a power dissipation of 9W out of the total 48W consumption. About 80% efficiency.

Not quite what I had hoped for… but this is a worst case scenario, with the nodes going flat chat and the battery charger pumping all the electrons it can. The lead acid battery has a nominal voltage of 13.8V… meaning we’re dropping 1.1V.

On a related note, I also overlooked this little paragraph in the motherboard handbook:

(*Do not use the 4-pin DC power @J1 when the 24-pin ATX Power @JPW1 is connected to the power supply. Do not plug in both J1 and JPW1 at the same time.)

Yep, guess what I’ve done. Being used to motherboards that provide both and needed both, I plugged them both in.

No damage done as all nodes work fine… (or they did last time I tried them… yet to fire them up since this last bit of surgery). It is possible there is no isolation between the on-motherboard PSU and the external ATX one and that if you did plug in power from two differing sources, you could get problems.

In a way if I had spotted this feature before, I could have done without those little PSUs after all, just needing a Molex-style power adaptor cable to plug into the motherboard.

Still… this works, so I’m not changing it. I have removed that extra connection though, and they’ve been disconnected from the PSUs so they won’t cause confusion in future.

I might give this a try when things cool down a bit … BoM still reports it being about 32°C outside (I have a feeling where I live is a few degrees hotter than that) and so I don’t feel energetic enough to drag my cluster out to the workbench just now. (Edit: okay, I know…those in NSW are facing far worse. Maybe one of the mob in New Holland should follow the advice of Crowded House and take the weather with them over here to the east coast! Not all of it of course, enough to cool us off and reduce their flood.)

Jan 292017

Well, I’ve finally dragged this project out and plugged everything in to test the new power modules out.

I’ll be hooking up the laptop and getting the nodes to do something strenuous in a moment, but for now, I have them just idling on the battery, with the battery charger being switched by the charge controller, built around an ATTiny24A and this time, a separate power module rather than it being integrated.

I’ve had it going for a few hours now… and so far, so good. The PSU is getting turned on and off more often than I’d like, but at least the smoke isn’t escaping now. The heatsink for the power modules is warm, but still not at the “burn your fingers off” stage.

That to me suggests the largish heatsink was the right one to use.

Two things I need to probably address though:

  • In spite of the LDOs, the acceptable voltage range of the computers is still rather narrow… I’m not sure if it’s just the IPMI BMC being fussy or if the LDOs need to be knocked down a peg to keep the voltage within limits. Perhaps I should use the same resistor values as I did for the Ethernet switch.
  • The thresholds seem to get reached very quickly which means the timeouts still need lengthening. Addressing the LDO settings should help with this, as it’ll mean I can bump my thresholds higher.

If I can nail those last two issues, then I might be at risk of having the hardware aspect of this project done and having a workable cluster to do the software side of the project. Shock horror!

Nov 052016

So, my last attempt at a fully integrated power controller was a smouldering failure. Q7 decided it wasn’t happy about where things were going, and let the world know by the only way it knew: smoke signals!

Curiously, only one of the MOSFETs in use seems to be damaged. When we look at its mate on the other side, Q2, sure it’s discoloured, which could be indication that it has been stressed, or maybe it just got burned by the other MOSFET.

It goes without saying that the pair in the background are fine: no current flowed through them during this test.

This got me thinking, did it get too hot, or did something else go wrong? This is the schematic and PCB layout of that part of the board.

Now looking at it, one thing strikes me. Seems I might have the source pins connected back-to-back, not the drain pins. Could that be it? I think I intended it the other way but didn’t pay enough attention to the schematic symbol. This could be a factor, or maybe the other MOSFET might’ve blown instead.

One thing is certain, I cannot join the two tabs together to the same heat-sink like I was intending with this schematic. Mia culpa!

Thinking about the design, the idea of putting the MOSFETs might be a tad naïve, as by far they are going to be the most likely component to fail on the board. The concept of a separate power module would work better since it’s easy then to just rip one out and put another in its place. Designed right, this could even be hot-pluggable, and can incorporate a heat-sink, and can make use of larger or smaller MOSFETs for different applications.

Luckily, the existing board layout will accommodate this just fine. We put 3-pin KK connectors in place of Q2 and Q4, and we can jumper across Q7 and Q8. This means the feed to the battery only needs to be a light-gauge wire, sufficient to power the controller and measure the battery voltage.

The pin-out isn’t ideal for this: it would be better to have a pin connecting to 0V instead of the battery +12V, but it’s workable. The gate pin becomes an open-collector output, and can theoretically drive (low-current) relays or MOSFETs.

As for what to build this power module on? Well, without going and buying a heat-sink, I’m spoiled for choice:

All of these dwarf the TO-220 package transistor I’m using… okay the one shown is an IRF-9540, but it’s still a TO-220 put there for scale reference. Most of these are for Intel CPUs that are long obsolete, and the top right has the CPUs in question still firmly attached.

The Pentium CPU heat-sink/fan would be the closest in the size, I was hoping I might’ve had a 486 heat-sink laying around, I’m of the opinion that if the power module needs a fan on any of these heat sinks, I’m doing it wrong. This might not be the case if I wanted the full 70A capability, but I’m pushing for 30 which is less than 50% capacity.

The only passive ones I have, and happen to have multiple of, are the ones on the lower right, which were extracted from dead Netgear (Bay Networks) switches. The BGA package still stuck to one of them is a Broadcom BCM5308A2KTB Ethernet switch SoC… it talked to a couple of SRAM packages (duly harvested) and a number of Ethernet PHYs.

The thought is that two MOSFETs could be fixed to the underside with a small PCB. (Well okay, there’s room for all four, but then I’ve got to somehow electrically insulate the two pairs.)

A connector of some sort, either a PCB edge connector, or perhaps a specially keyed Andersen Power Pole connector pairs (which can be rotated 90°) could connect power and control in one secure mounting. Two 30A connectors and a 15A would serve this job well, and they come in a range of colours for the housings. Thus I can avoid the red/black typical colouring to avoid confusion.