ti-ina219b

Solar Cluster: Re-wiring the rack

It’s been on my TO-DO list now for a long time to wire in some current shunts to monitor the solar input, replace the near useless Powertech solar controller with something better, and put in some more outlets.

Saturday, I finally got around to doing exactly that. I meant to also add a low-voltage disconnect to the rig … I’ve got the parts for this but haven’t yet built or tested it — I’d like to wait until I have done both,but I needed the power capacity. So I’m running a risk without the over-discharge protection, but I think I’ll take that gamble for now.

Right now:

  • The Powertech MP-3735 is permanently out, the Redarc BCDC-1225 is back in.
  • I have nearly a dozen spare 12V outlet points now.
  • There are current shunts on:
    • Raw solar input (50A)
    • Solar controller output (50A)
    • Battery (100A)
    • Load (100A)
  • The Meanwell HEP-600C-12 is mounted to the back of the server rack, freeing space from the top.
  • The janky spade lugs and undersized cable connecting the HEP-600C-12 to the battery has been replaced with a more substantial cable.

This is what it looks like now around the back:

Rear of the rack, after re-wiring

What difference has this made? I’ll let the graphs speak. This was the battery voltage this time last week:

Battery voltage for 2019-05-22

… and this was today…

Battery voltage 2019-05-29

Chalk-and-bloody-cheese! The weather has been quite consistent, and the solar output has greatly improved just replacing the controller. The panels actually got a bit overenthusiastic and overshot the 14.6V maximum… but not by much thankfully. I think once I get some more nodes on, it’ll come down a bit.

I’ve gone from about 8 hours off-grid to nearly 12! Expanding the battery capacity is an option, and could see the cluster possibly run overnight.

I need to get the two new nodes onto battery power (the two new NUCs) and the Netgear switch. Actually I’m waiting on a rack-mount kit for the Netgear as I have misplaced the one it came with, failing that I’ll hack one up out of aluminium angle — it doesn’t look hard!

A new motherboard is coming for the downed node, that will bring me back up to two compute nodes (one with 16 cores), and I have new 2TB HDDs to replace the aging 1TB drives. Once that’s done I’ll have:

  • 24 CPU cores and 64GB RAM in compute nodes
  • 28 CPU cores and 112GB RAM in storage nodes
  • 10TB of raw disk storage

I’ll have to pull my finger out on the power monitoring, there’s all the shunts in place now so I have no excuse but to make up those INA-219 boards and get everything going.

2019/05/29 by Computing Projects Public Syndication Solar-powered Cloud Computing 0

DC-DC Converter: Sensor board re-designed

So, I placed an order to Mouser the other day to actually get some parts into my hands so I can better design the boards.

In that, I discovered the screw terminals I was planning on using, are discontinued.  So, I found something that was able to take the same gauge wire: Phoenix 1017526s.  Turns out, these will not fit along side the current shunts on the board as planned.

There’s just no way I’ll be fitting these on a 5×5cm board and have room to spare for a shunt in between.  Since this is really application-specific, it might be better off board.  We’ll put the INA219 and PCA9615 together on the board so we have a nice self-contained sensor board that can be mounted close to the current shunt, wherever that lives, and have nice noise-resistant links back to the controller.

This does mean I can do things like put a current shunt in the fuse box where the solar panels connect, and run CAT5 down to the controller from there.

To make routing easier, I’ve gone to a 4-layer board.  The board has solder-jumpers for setting the I²C address of the INA219, and I’ve documented all the termination and pull resistors.  I’m not sure what ones are needed yet, so there’s space at every point where I could envisage one being needed.

There’s two power planes in the inner layers, one for VCC the other for 0V.

Next step, I’ll print out the board designs and test fit everything before ordering the boards, which I hope to have ordered this afternoon.

2018/10/21 by High-power DC-DC Converter Projects Public Syndication 0

DC-DC Converter: Sensor board designed

So, I’ve designed the sensor board, this is basically a break-out of an INA219 coupled with a PCA9615, for extended I²C range.  If I was to use one of these on the cluster, it’s theoretically possible for me to put one up in the fuse box on the back deck, and run CAT5e down to the server rack to help measure voltage drop across that long run.  Doing that with regular I²C would be insane.

Again, I’ve gone crazy with pull-up, pull-down and termination resistances, not knowing what would be needed.  The schematic is nothing special.

The board wound up bigger than I’d expected, but largely because it had to accommodate fairly heavy power traces.  I think I’ve got the footprint for the screw terminal blocks right.  I’ve managed to cram it onto a 5cm×5cm board (two layer).

As always, you’ve got two ways of dealing with the current shunt, either hook one up externally, which means you don’t bother with the beefy power connection footprints, or you fit a surface-mount shunt on.

You’ve got full flexibility there, as well as what address to set the board to via the jumpers.

I’ll probably order some of the connectors and other parts in question, print out the board layout and test-fit everything.  I’m not happy about the fact that NXP only make the PCA9615 in TSSOP, but I guess I should be thankful the part has legs.

2018/10/06 by High-power DC-DC Converter Projects Public Syndication 0

DC-DC Converter: Controller designed

So this is what I’ve come up with for the core controller.

There’s provisions for two versions on this board, one with an ATTiny861 which does high-speed (250kHz) PWM and can drive a buck, boost or buck-boost DC-DC converter.  It features differential I²C interfaces for the input and output INA219 boards, and LVDS for controlling the MOSFET boards.

The other version is built around the ATTiny24A, and just features the ability to turn on and off MOSFETs.  It can drive two statically, or PWM one (at a much lower speed), with the user supplying the driver logic.  Due to the the fact that this device does not do high-speed switching, I’ve forgone the LVDS control over a simple current loop.  The I²C is still differential though as that could be some distance away and is still somewhat high frequency.

The layout of the board is a small 5×5cm 4-layer PCB.

I had to go 4-layer as I needed to route signals both sides and didn’t want to interrupt the power planes.  The two inner layers are VCC and GND.  There’s de-coupling capacitors galore, although the two power planes will probably function as a decent capacitor in their own right.  ICSP is via the interface header at the bottom.

2018/09/30 by High-power DC-DC Converter Projects Public Syndication 0

DC-DC Converter: Splitting up the project

I was originally thinking of one monolithic board which would have everything it needed.

There was provision for the lot, including a separate ATTiny24A so that you could omit all but one of the MOSFETs, swap the remaining MOSFET for a P-channel, drop the MOSFET drivers, one of the INA219s, and the ATTiny861, and you’d have just a monitoring board with a (low-speed) PWMable switch.  It’d plug into the same place and use the same host interface.  The one board could be made into just a boost, or just a buck.  Flexibility.

There was just one snag.  That’ll work for small power supplies with maybe up to 5A capability (~100W) but not for the 50A version.  The MOSFETs will fit, but the tracks will need to be huge, the board will be hideously expensive to make, and they don’t make inductors big enough.

Looking around for inductors, I did see these.  They’re not massive like the 10uH one I saw, and they’re not expensive.  The downside is they’re about 10% of what I really need.  I guess I’ll just make do.  They’re also not PCB-mount (mind you, a 40kg inductor doesn’t PCB-mount either).

Thus, it may be more sensible to separate the MOSFETs and high-power stuff from the controller.  Now here’s the rub.  We’re dealing with sub-15ns pulses.  PWM.

Years ago for my industrial experience, I did work on an electric harvester platform.  The system ran 48V.  The motors were rated at 20kW, and were made in house using windings wound from 5mm diameter enamelled copper wire and neodymium magnets.

We had loads of issues with MOSFETs blowing.  The MOSFET driver was mounted close to the MOSFETs, as I’m proposing to do here, but between DSP and driver, was a long-ish run of ribbon cable. @Bil Herd posted this article covering the challenges involving inductance on PCB layout.  That same problem applies to “long” cable runs too.

10 years ago when we were working on this project, I remember asking about if we had considered maybe using coax cable instead of a ribbon cable.  The idea was rubbished at the time.  Given we were PWMing 400A, I think there might’ve been something in that suggestion.

That ribbon (10~20cm of it) would have had a lovely inductance all of its own, and while I have no idea what frequency the PWM was running at (I might have the code somewhere but I can’t be stuffed digging it up), and we were fundamentally driving a single-ended signal over a fairly long distance.  Yes, ground was close, but probably not enough, a twisted pair would have been better, but even then not perfect.  We blew many MOSFETs on that project.  Big TO-263s!

An earlier article on differential signalling got me thinking: why not use LVDS for the PWM?  A quick search has revealed this receiver and transmitter (Mouser says two receivers on it, but I think that’s a typo).  The idea being that I send the PWM down a differential pair using LVDS.  155Mbps should be plenty fast enough (the ATTiny861 can only do 64MHz) and these parts will run at the 5V needed for fast switching.  In fact they require it.

Using twisted pairs, the inductance should cancel.  I’ll make a MOSFET board that just has these signal pairs:

  • +12V (for the MOSFET driver) and 0V
  • +5V (for the LVDS receiver) and 0V
  • High side PWM + and –
  • Low side PWM + and –

There’s a ground-loop I need to be wary of between the 12V and 5V rails, really it’s the same 0V rail for both.  I suspect they’ll still need to be connected at both ends.  Add in more of those screw terminals to take the input and output power off-board, and I think we should be set.

Similarly, the INA219 should probably be a separate board, with scope for having a chassis-mounted current shunt.  The connection to the current shunt’s sense output is a low-power connection, so no issue there.  You want to keep it short for accuracy reasons, but a simple twisted pair will work fine.

2018/09/30 by High-power DC-DC Converter Projects Public Syndication 0

Solar Cluster: Jury still out on solar controller, thinking of PSU designs

So, the last few days it’s been overcast.  Monday I had a firmware glitch that caused the mains supply to be brought in almost constantly, so I’d disregard that result.

Basically, the moment the battery dropped below ~12.8V for even a brief second, the mains got brought in.  We were just teetering on the edge of 12.8V all day.  I realised that I really did need a delay on firing off the timer, so I’ve re-worked the logic:

  • If battery drops below V_L, start a 1-hour timer
  • If battery rises above V_L, reset the 1-hour timer
  • If the battery drops below V_CL or the timer expires, turn on the mains charger

That got me better results.  It means V_CL can be quite low, without endangering the battery supply, and V_L can be at 12.8V where it basically ensures that the battery is at a good level for everything to operate.

I managed to get through most of Tuesday until about 4PM, there was a bit of a hump which I think was the solar controller trying to extract some power from the panels.  I really need a good sunny day like the previous week to test properly.

This is leading me to consider my monitoring device.  At the moment, it just monitors voltage (crudely) and controls the logic-level enable input on the mains charger.  Nothing more.  It has done that well.

A thought is that maybe I should re-build this as a Modbus-enabled energy meter with control.  This idea has evolved a bit, enough to be its own project actually.  The thought I have now is a more modular design.

If I take the INA219B and a surface-mount current shunt, I have a means to accurately measure input voltage and current.  Two of these, and I can measure the board’s output too.  Stick a small microcontroller in between, some MOSFETs and other parts, and I can have a switchmode power supply module which can report on its input and output power and vary the PWM of the power supply to achieve any desired input or output voltage or current.

The MCU could be the ATTiny24As I’m using, or a ATTiny861.  The latter is attractive as it can do high-speed PWM, but I’m not sure that’s necessary in this application, and I have loads of SOIC ATTiny24As.  (Then again, I also have loads of PDIP ATTiny861s.)

The board would expose the ICSP pins plus two more for interrupt and chip select, allowing for a simple jig for reprogramming.  I haven’t decided on a topology yet, but the split-pi is looking attractive.  I might start with a buck converter first though.

This would talk to a “master” microcontroller which would provide the UI and Modbus interface.  If the brains of the PSU MCU aren’t sufficient, this could do the more grunty calculations too.

This would allow me to swap out the PSU boards to try out different designs.

2018/09/27 by Computing Projects Public Syndication Solar-powered Cloud Computing 0

Solar Cluster: TS-7670 showed up today

So, I now have my little battery monitoring computer.  Shipping wound up being a little more than I was expecting… about US$80… but never mind.  It’s here, arrived safely:

HTLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLFLC
>> TS-BOOTROM - built Jan 26 2017 12:29:21
>> Copyright (c) 2013, Technologic Systems
LLCLLLLLLLFLCLLJUncompressing Linux... done, booting the kernel.
/ts/fastboot file present.  Booting to initramfs instead
Booted from eMMC in 3.15s
Initramfs Web Interface: http://ts7670-498476.local
Total RAM: 128MB
# exit
INIT: version 2.88 booting
[info] Using makefile-style concurrent boot in runlevel S.
[ ok ] Starting the hotplug events dispatcher: udevd.
[ ok ] Synthesizing the initial hotplug events...done.
[ ok ] Waiting for /dev to be fully populated...done.
[ ok ] Activating swap...done.
[....] Checking root file system...fsck from util-linux 2.20.1
e2fsck 1.42.5 (29-Jul-2012)
/dev/mmcblk2p2: clean, 48540/117600 files, 282972/469760 blocks
done.
[ ok ] Cleaning up temporary files... /tmp /lib/init/rw.
…
ts7670-498476 login: root
Linux ts7670-498476 2.6.35.3-571-gcca29a0+ #1 PREEMPT Mon Nov 27 11:05:10 PST 2017 armv5tejl
TS Root Image 2017-11-27

The programs included with the Debian GNU/Linux system are free software;
the exact distribution terms for each program are described in the
individual files in /usr/share/doc/*/copyright.

Debian GNU/Linux comes with ABSOLUTELY NO WARRANTY, to the extent
permitted by applicable law.

root@ts7670-498476:~#

The on-board 2GB eMMC has a version of Debian Wheezy on it.  That’ll be going very soon.  For now, all I’ve done is pop the cover, shove a 8GB MicroSD card into one of the on-board slots, wired up a 12V power brick temporarily to the unit, hooked a USB cable into the console port (/dev/ttyAMA0 is wired up to an on-board CP2103 USB-serial chip) and verified that it is alive.

Next step will be to bootstrap Gentoo.  I could use standard ARMv5 stages, or I can build my own, which I might do.  I’ve done this before for mips64el n64 using glibc.  Modern glibc is a goliath on a machine with 128MB RAM though, so I’ll be looking at either µClibc/µClibc-ng or musl… most likely the latter.

That said, 20 years ago, we had the same computing power in a desktop. 🙂

I have a few options for interfacing to the power meters…

  • I²C, SPI, a number of GPIOs and a spare UART on a 2.54mm header inside the case.
  • Another spare UART on the footprint for the GPS module (which my unit does not have)
  • Two RS-232 serial ports with RTS/CTS control lines, exposed via RJ-45 jacks
  • Two CANbus ports on a single RJ-45 jack
  • RS-485 on a port marked “Modbus”

In theory, I could just skip the LPC810s and hook this up directly to the INA219Bs.  I’d have to double check what the TTL voltage is… Freescale love their 1.8V logic… but shifting that up to 3.3V or 5V is not hard.  The run is a little longer than I’m comfortable running I²C though.

The LPC810s don’t feature CANbus, so I think my original plan of doing Modbus is going to be the winner.  I can either do a single-ended UART using a resistor/diode in parallel to link RX and TX to the one UART line, or use RS-485.

I’m leaning towards the latter, if I decide to buy a little mains energy meter to monitor power, I can use the same RS-485 link to poll that.  I have some RS-485 transceivers coming for that.

For now though, I’ll at least get Debian Stretch going… this should not be difficult, as I’ll just use the images I’ve built for work to get things going.  I’m downloading a Jessie image now:

root@ts7670-498476:~# curl https://bne.vrt.com.au/technologicsys/ts7670d-jessie-4.4.1-20160226.dd.xz | xzcat | dd of=/dev/mmcblk0 
  % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current
                                 Dload  Upload   Total   Spent    Left  Speed
  0  113M    0  544k    0     0   114k      0  0:16:48  0:00:04  0:16:44  116k

Once that is done, I can reboot, re-format the eMMC and get debootstrap going.  I might even publish an updated image while I’m at it.

2018/01/24 by Computing Linux Development Projects Public Syndication Solar-powered Cloud Computing 0

Solar Cluster: 2 days and counting on solar…

So, I’m home now for the Christmas break… and the fan in my power supply decided it would take a Christmas break itself.

The power supply was purchased brand new in June… it still works as a power supply, but with the fan seized up, it represents an overheating risk.  Unfortunately, the only real options I have are the Xantrex charger, which cooked my last batteries, or a 12V 20A linear PSU I normally use for my radio station.  20A is just a touch light-on, given the DC-DC converter draws 25A.  It’ll be fine to provide a top-up, but I wouldn’t want to use it for charging up flat batteries.

Now, I can replace the faulty fan.  However, that PSU is under warranty still, so I figure, back it goes!

In the meantime, an experiment.  What happens if I just turn the mains off and rely on the batteries?  Well, so far, so good.  Saturday afternoon, the batteries were fully charged, I unplugged the mains supply.  Battery voltage around 13.8V.

Sunday morning, battery was down to 12.1V, with about 1A coming in off the panels around 7AM (so 6A being drained from batteries by the cluster).

By 10AM, the solar panels were in full swing, and a good 15A was being pumped in, with the cluster drawing no more than 8A.  The batteries finished the day around 13.1V.

This morning, batteries were slightly lower at 11.9V.   Just checking now, I’m seeing over 16A flowing in from the panels, and the battery is at 13.2V.

I’m in the process of building some power meters based on NXP LPC810s and TI INA219Bs.  I’m at two minds what to use to poll them, whether I use a Raspberry Pi I have spare and buy a case, PSU and some sort of serial interface for it… or whether I purchase a small industrial PC for the job.

The Technologic Systems TS-7670 is one that I am considering, given they’ll work over a wide range of voltages and temperatures, they have plenty of UARTs including RS-485 and RS-232, and while they ship with an old Linux kernel, yours truly has ported both U-Boot and the mainline Linux kernel.  Yes, it’s ARMv5, but it doesn’t need to be a speed demon to capture lots of data, and they work just fine for Barangaroo where they poll Modbus (via pymodbus) and M-bus (via python-mbus).

2017/12/25 by Computing Linux Development Projects Public Syndication Solar-powered Cloud Computing Thinktank 0

Solar Cluster: Solar back on…

So, this weekend I did plan to run from solar full time to see how it’d go.

Mother nature did not co-operate.  I think there was about 2 hours of sunlight!  This is what the 24 hour rain map looks like from the local weather radar (image credit: Bureau of Meteorology):

In the end, I opted to crimp SB50 connectors onto the old Redarc BCDC1225 and hook it up between the battery harness and the 40A power supply. It’s happily keeping the batteries sitting at about 13.2V, which is fine. The cluster ran for months off this very same power supply without issue: it’s when I introduced the solar panels that the problems started. With a separate controller doing the solar that has over-discharge protection to boot, we should be fine.

I also have mostly built-up some monitoring boards based on the TI INA219Bs hooked up to NXP LPC810s. I have not powered these up yet, plan is to try them out with a 1ohm resistor as the stand-in for the shunt and a 3V rail… develop the firmware for reporting voltage/current… then try 9V and check nothing smokes.

If all is well, then I’ll package them up and move them to the cluster. Not sure of protocols just yet. Modbus/RTU is tempting and is a protocol I’m familiar with at work and would work well for this application, given I just need to represent voltage and current… both of which can be scaled to fit 16-bit registers easy (voltage in mV, current in mA would be fine).

I just need some connectors to interface the boards to the outside world and testing will begin. I’ve ordered these and they’ll probably turn up some time this week.

2017/11/19 by Computing Projects Public Syndication Solar-powered Cloud Computing 0

Solar Cluster: Suspicions with the charger…

So, at present I’ve been using a two-charger solution to keep the batteries at full voltage.  On the solar side is the Powertech MP3735, which also does over-discharge protection.  On the mains side, I’m using a Xantrex TC2012.

One thing I’ve observed is that the TC2012, despite being configured for AGM batteries, despite the handbook saying it charges AGM batteries to a maximum 14.3V, has a happy knack of applying quite high charging voltages to the batteries.

I’ve verified this… every meter I’ve put across it has reported it at one time or another, more than 15V across the terminals of the charger.  I’m using SB50 connectors rated at 50A and short runs of 6G cable to the batteries.  So a nice low-resistance path.

The literature I’ve read says 14.8V is the maximum.  I think something has gone out of calibration!

This, and the fact that the previous set-up over-discharged the batteries twice, are the factors that lead to the early failure of both batteries.

The two new batteries (Century C12-105DA) are now sitting in the battery cases replacing the two Giant Energy batteries, which will probably find themselves on a trip to the Upper Kedron recycling facility in the near future.

The Century batteries were chosen as I needed the replacements right now and couldn’t wait for shipping.  This just happened to be what SuperCheap Auto at Keperra sell.

The Giant Energy batteries took a number of weeks to arrive: likely because the seller (who’s about 2 hours drive from me) had run out of stock and needed to order them in (from China).  If things weren’t so critical, I might’ve given those batteries another shot, but I really didn’t have the time to order in new ones.

I have disconnected the Xantrex TC2012.  I really am leery about using it, having had one bad experience with it now.  The replacement batteries cost me $1000.  I don’t want to be repeating the exercise.

I have a couple of options:

  1. Ditch the idea of using mains power and just go solar.
  2. Dig out the Redarc BCDC1225 charger I was using before and hook that up to a regulated power supply.
  3. Source a new 20A mains charger to hook in parallel with the batteries.
  4. Hook a dumb fixed-voltage power supply in parallel with the batteries.
  5. Hook a dumb fixed-voltage power supply in parallel with the solar panel.

Option (1) sounds good, but what if there’s a run of cloudy days?  This really is only an option once I get some supervisory monitoring going.  I have the current shunts fitted and the TI INA219Bs for measuring those shunts arrived a week or so back, just haven’t had the time to put that into service.  This will need engineering time.

Option (2) could be done right now… and let’s face it, its problem was switching from solar to mains.  In this application, it’d be permanently wired up in boost mode.  Moreover, it’s theoretically impossible to over-discharge the batteries now as the MP3735 should be looking after that.

Option (3) would need some research as to what would do the job.  More money to spend, and no guarantee that the result will be any better than what I have now.

Option (4) I’m leery about, as there’s every possibility that the power supply could be overloaded by inrush current to the battery.  I could rig up a PWM circuit in concert with the monitoring I’m planning on putting in, but this requires engineering time to figure out.

Option (5) I’m also leery about, not sure how the panels will react to having a DC supply in parallel to them.  The MP3735 allegedly can take an input DC supply as low as 9V and boost that up, so might see a 13.8V switchmode PSU as a solar panel on a really cloudy day.  I’m not sure though.  I can experiment, plug it in and see how it reacts.  Research gives mixed advice, with this Stack Exchange post saying yes and this Reddit thread suggesting no.

I know now that the cluster averages about 7A.  In theory, I should have 30 hours capacity in the batteries I have now, if I get enough sun to keep them charged.

This I think, will be a week-end experiment, and maybe something I’ll try this weekend.  Right now, the cluster itself is running from my 40A switchmode PSU, and for now, it can stay there.

I’ll let the solar charger top the batteries up from the sun this week.  With no load, the batteries should be nice and full, ready come Friday evening, when I can, gracefully, bring the cluster down and hook it up to the solar charger load output.  If, at the end of the weekend, it’s looking healthy, I might be inclined to leave it that way.

2017/11/13 by Computing Projects Public Syndication Solar-powered Cloud Computing 0

Calendar

March 2023
S M T W T F S
 1234
567891011
12131415161718
19202122232425
262728293031  

Battery voltage

Mastodon

Mastodon