I initially got a BSEE degree, and started designing logic circuits out of school. After a couple of years, I realized most of what I was doing could be done using microprocessors (this was back in the 1970's, when the first ones were just coming on the market). So I went back and got an MSCS degree. Ever since then I have leveraged the combination of the two in a successful career of embedded hardware design and programming.
Without some formal training, it is pretty hard to pick up enough electronics knowledge to design complex circuits from scratch; from my experience it's always seems easier to teach a EE to program than teach a CS major to design hardware.
That said, you can probably learn enough on your own to understand schematics of development boards etc., at least the digital portions. Analog circuitry (op-amps, amplifiers, power supplies etc.) is more of a black art, especially high-frequency RF (radio frequency) circuits. There's a lot of math involved; for example check out these equations for various kinds of filters. This is a far cry from digital logic where one works with Boolean algebra and things like Karnaugh maps. Since you're not going to be designing circuits, you just need a basic understanding of logic gates (AND, OR, NAND, NOR, XOR and NOT). You may also want to learn about flip-flops (which are used to make registers), multiplexers and de-multiplexers. These building blocks, along with memory, make up the guts of a microcontroller.
A book I highly recommend to start with is "Practical Electronics for Inventors"; forget the Inventors angle, it just plainly a good basic book on the subject. (Another book often mentioned is "The Art of Electronics", but it costs four times as much ($90 vs $22) and is quite a bit more theoretical.
Another book you might look into is "Code: The Hidden Language of Computer Hardware and Software", by Charles Petzold, who was a legend at Microsoft. It is fairly basic, but does do a nice job of showing how the hardware and software work together.
In terms of tools, you're going to want to get at least a decent multimeter, that can measure voltage, current, resistance, and maybe capacitance. One of the satisfying first steps in embedded programming is to be able to blink an LED using one of the port pins on a microcontroller. This is essentially the "Hello World" of the embedded world. You will find dozens if not hundreds of posts on this forum re this activity.
Eventually, if you want to get serious about poking around circuits, you will want to get a digital oscilloscope, perhaps 60 or 100 MHz.
As someone else said, one of the skills you will want to master is reading datasheets for the various parts on a PCB (printed circuit board); not just the microcontroller but also various peripherals like sensors, LCDs, modems (used for cellular, Bluetooth, and other communications), etc.
Since you have already worked with an Arduino, I suggest you step up from the 8-bit processor world to a 32-bit one. You will want to pick a particular microprocessor family and then get a development board to go with it. I'm going to suggest an ARM Cortex-M7 processor: the STM32F746NGH6. It is a RISC processor, like AVR32 and MIPS, running at 216 MHz. The processor has 1MB of Flash, 340K of RAM, and all the usual serial interfaces (I²C, SPI, UART, USB) plus up to 18 timers. You may or may not have encountered some of these peripherals on the Arduino. You will find that all of the hardware interfacing is done by writing into dedicated registers on the chip provided for each peripheral. Explaining all of these is what makes the datasheets so long (220 pages for this particular chip). You can obviously ignore some of the more complicated stuff like DMA and DSP until later.
There is a very nice evaluation board available for $50. It has a 4.3" touch screen LCD, 8 MB of additional RAM and lots of other goodies. In any case, you want to stay away from single board computers (SBCs) like the Raspberry Pi and Beaglebone Black; they are very capable and also inexpensive (you can get a Raspberry Pi for a little as $20). However they in general run Linux, and it is much tougher to get "down to the metal" on those because you have to deal with device drivers that fit into the Linux model.
I strongly suggest you try to write some assembly language programs, in addition to C, as it will definitely give you a better idea of what is going on under the hood. Plus it's just fun. The book "ARM7 Assembly Language Programming: 100+ examples" is only available as an eBook (Kindle), but you can download a Kindle reader for your PC if you don't own a Kindle.
If you program only in C (or some other high-level language such as C++, but the vast majority of firmware for microcontrollers is in C), you won't see any references to the processor's registers. You need to either program in assembly, or have a disassembler available where you can see the machine code the compiler is generating for you.
In addition, the book "Digital Design and Computer Architecture: ARM Edition" looks interesting since it covers both hardware design and ARM architecture. So you may be able to just use it as your initial hardware introduction. (You can click on "Look inside" for an of these books to see the Table of Contents and sample pages.)
