Ttdfjt

Background:

Junior R&D electronics engineer (the only EE in the company) -
the hardware and the coding is not the problem. My biggest issue is getting a proper overview of the project, and where to start.

So far I've only made minor software projects (sub 500 lines of code), but I can't envision myself doing larger projects without losing overview of the functionality or lack of functionality.

How do you best structure / what tools do you use to structure large embedded software systems?

What I'm currently doing:

I usually start out, by sketching the functionality of the project.
It could be one to many layered flow charts or related diagrams (block diagrams, etc.) and doing some research of the components/chips. Then I jump straight into coding (fail fast I guess) while referencing the datasheets / Internet,
Coding one functionality at a time and testing it with dummy data, or similar test. It could be writing data to a MEM chip, and then if that works then
it could be an SPI driver between the main chip and the MEM chip.

What answer I'm looking for:

Anything really. I will sort out what I find sensible. It could be a book, an article, personal experience, recommendations, etc.

I'm very interested in knowing how seniors tackle this.

Edit

First off, thank you for sharing your years of experience! All the answers are much appreciated. My take from this is;

• Create a clear and precise specification document.


• Create a software design document. (Something I will now add)
  Design doc templates
  


• Think in modules how ever redundant it may seem. (Something I need to focus more on)


• Follow a coding standard for structuring header/source files. (Never did this)
  Barr Group C standard
  


• Focus on creating the low level implementations first. (Communication etc.)


• Implement design patterns where ever possible/sensible.
  Design patterns
  


• Set up something for revision control (Github etc. - never used this much)


• Research continuous integration / continuous deployment (Something new I stumbled upon)
  CI & CD basics
  

There are several aspects influencing the grade of detail the structuring of a project needs. For me one of the main factors is whether I'm the only one coding (what seems to be the case for you as you write you're the only EE) or if there are others involved.
Then there's the question of what "large" actually means. Usually I divide the design process into the following steps:

Requirement definition
If you get proper software specification to work with a lot of planning is already done. If you just get vague requirements, the first thing you have to do is to sort out what the customer actually wants (sometimes they don't really know in the first place). I know it's tempting to just jump right into coding, but that brings the risk of missing an important feature that might was not obvious in the first place and can't just be easily squeezed into your code just somewhere in the middle of development.

System boundaries and maintainability
In embedded systems you often have some system interfaces, some to the outside (operator) but also on the inside. Define these well and try to keep dependencies as low as possible, this will simplify continuous engineering and maintainability. Also comment/document code where needed, you never know who else will have to work with it, (s)he will be happy to not have to dig though a dozen layers of code before actually knowing what a function does.

Define verifiable tasks
Especially if other developers are working on the same code base it's inevitable to define clear tasks (features) and the required interfaces between them. Whenever possible the individual features should be tested/verified independent from others, that's where you need the interfaces well defined so you can define your test cases.

One feature after the other
People like progress, so if you have a variety of tasks they usually work on whatever promises the most progress. I usually try to finish a task and bring it to a verified and tested state before I start with the next one. This allows your code to be tested by others and you not ending up forgetting something.

Revision Control
During the life of a project you sometimes need older versions, maybe to identify a bug introduced with some new release or just to build a device that behaves exactly the same way as one you shipped 3 years ago. Make sure you have clear build revisions and tags in your code. Git is definitely your friend here.

Humpawumpa wrote a great answer! I just want to supplement some of his points, but since this is too long to be a comment, I'll write a separate answer.

I was once in the OP's position — not the only EE, but the only EE who had done any MCU development in a small company.

I can't emphasize the importance of modularity enough, even if you're the only developer. It's the only way to stay sane as the project grows. You need to be strict about writing modules that handle only one functional concept each, and keep their external interfaces as minimal as possible. High-level modules will correspond to functional requirements, while low-level modules will have close ties to hardware resources (i.e., device drivers).

I spent a lot of time maintaining a detailed data flow diagram¹, which showed precisely how the various modules shared information. Some features will have very different requirements in terms of real-time performance; make sure you know how the information sharing affects that. The diagram had boundaries drawn across it that separated the non-interrupt code from the various interrupt-driven domains.

¹ Very different from a flowchart, which is focused on control flow.

For any large project, I plan it as if there were multiple developers involved even if I intend to do the whole thing myself.

The reasons are simple:

1 Complexity. A large project will always have complexities involved. Having planned the project as if there were multiple teams involved means that complexity has been considered and documented. The number of times I have seen large projects run into problems is high and usually because something 'slipped through the cracks'. Do not forget that mechanical assembly must also be considered and not simply for the size of the box - will there be a need for heat sinks? Must the box be earthed for safety? There are many questions in this category alone.

2 Requirements. Assuming multiple people are involved means that the top level requirements (which I often challenge as they may bring unnecessary complexity and cost) must be broken down into the various smaller required and achievable tasks (which could be fed to a another team in a larger organisation) rather than just looking at a single blob.

3 Partitioning. There are two primary types of partitioning; hardware functionality and hardware / software. The first type is to determine what separate (but communicating) functional blocks need to be present. The second is a trade-off of simpler (sometimes) hardware and software but keep in mind that simply moving more things to software will not necessarily fix a hardware problem. Moving more into software can actually vastly increase hardware complexity in some circumstances (more processing horsepower, more complex interfaces and more).

4 Interfaces. The partitioning process will help define internal interfaces; external interfaces are usually part of the overall requirements (although not always). There are many ways for different parts of a system to co-operate which may be a complex communications protocol or simply good / bad signalling.

5 Verification. This is a mixture of low level testing (for hardware and drivers) and system level. Having done the project in well defined blocks permits robust verification (which may be by analysis or actual test and is usually a mixture of the two; updates to designs may use similarity arguments).

6 Standards. I use coding and drawing standards as if it were a larger team. I use Barr group's coding standards as they are not too onerous but do prevent many classes of bug being in the code. For PCB output documentation I follow IPC-D-326 (which calls up IPC-D-325) as that is a proven method of communicating my intent to PCB fabricators and assemblers. This may seem strange but having the discipline to follow a number of standards means that the quality is consistent.

7 Version control. I use revision control for all parts of the project (system, hardware, software. mechanical, test requirements - everything). The CAD tools I use support such versioning as do all the software and FPGA build tools.

I have worked on many embedded projects (as have many of the experienced folk around here) and the team sizes have varied from 1 (myself) to dozens (or hundreds on a particular set of projects) spread across multiple disciplines and sometimes other geographically remote sites. Using the same over-arching approach (that is known to work) means I can pick up a particular task in relative isolation and complete it and thoroughly test it as a stand-alone part of the larger project. It also means I can sub some things out on occasion if necessary.

Doing all these things adds a bit of up-front time but is ultimately a faster route for complex embedded systems.

The other answers give many great tips. Here are two that I've found the most important in my embedded development career:

1. Make as much of the code into separate, well defined modules as possible.


2. Make the modules automatically testable on PC.

That's what you need for doing "continuous integration" style development on embedded systems. There will always be some amount of code that is too tightly tied to hardware for automatic testing, but try to minimize it. You can get quite far by using simulated data or data captures from actual hardware, which you then feed into the test system.

To add to existing answers...

I always start from the bottom up. From your hardware design, you know what your I/O is. Start by building driver modules which encapsulate that I/O, so that your high level code doesn't have to know too much about the low level stuff.

When you're building the low level interfaces, of course you need a test harness. If you design this to hook up to the PC from the start (perhaps with an RS-232 port, perhaps USB, perhaps telnet over Ethernet) then you can keep this test harness interface in place as you build your application. You can keep adding more test harness hooks as the application takes shape, and that'll let you regression-test your code as you go on as well.

I tend to think in four questions. The first two belong to the start of a systems project, the two next towards the end.

1. Do they actually want the system? Will the system solve the customers problem, at a time and cost scale they will accept? A common problem is building systems that the customer will not use.




2. Can we actually build that system? Will it be possibly to deliver the necessary performance, precision, power usage, ... ?

Creating early prototypes is a good way to answer these two first questions. Risk mitigation is extremely important in the early phases.

The next two questions are more targeted at the later phases of the the project:

3. are we actually finished? Everything designed, coded, tested, delivered




4. Do they actually use the system?