Dual THSCU101 with Raspberry Pi on LEGO Crane

Introduction

Summer camp season has started! Playing with exciting products during break time often provides great inspiration not only to kids but people working with kiddy mind!

Using following products, we built a proof-of-concept vision system for crane-like applications.

• LEGO® Mobile Crane #42108

• Raspberry Pi 400 computer

• 2 x THine Solutions THSCU101

In crane(-like) applications where the object (the hook) moves in 3 dimensions, one camera is not sufficient. For example, in the Forward Camera View, you can see the height of the hook and how far it needs to go in the right to hook up the cart, but you cannot tell if it is above it. By looking at the Top Camera View, now you can tell the hook needs to go a bit farther, but you need to get back to the other view to move the hook down to the cart. And you would notice fixed focus camera is not helpful enough as the object goes back and forth. Thus, 2 quick Auto Focus camera views enable your accurate operation.

This page will give you how we built the system, so you can get inspired for your next project or your kid’s summer camp extra.

Hardware Overview

LEGO Mobile Crane

It seems this product (#42108) is a bit outdated, but you may be able to find one in Amazon. (There are many other LEGO crane trucks!)

You would need some more parts to install THSCU101 THEIA-CAM™ to the Crane. (See following pictures.)

THine Solutions THEIA-CAM™ THSCU101

The THSCU101 is available at Amazon and Digi-Key. A USB 3.0 Micro-B to Type A cable is included in the THSCU101 Kit.

Here are some pictures for the Top Camera. The S-Mount hole of THSCU101 matches with LEGO Technic axles. (Sometimes it is a bit tight though.)

Forward Camera may need you to try a bit more. It needs to look a bit down rather than looking up, and horizontal direction tweak may be needed unless the camera is right under the Crane Arm.

Raspberry Pi 400

One of the great features the Raspberry Pi 400 has is 2 ports of USB 3.0, which allow you to use 2 different high resolution cameras. All you would need additionally is HDMI monitor.

We will now show you how to easily stream 2 different 4K camera videos with Phase Detection Auto Focus on the Raspberry Pi computer display!

Driver Installation

If you have set up THSCU101 on your Raspberry Pi, you can skip this section and go to “Start Streaming.”

Download the File

The ‘ignition key’ is here. Place the zip file “THEIA-CAM.zip“ on Desktop of the Raspberry Pi.

Install the Drivers

Unzip and place the folder in Desktop by launching the Terminal window and executing following commands:

cd Desktop

unzip THEIA-CAM.zip

ls

Now you see a folder on the Desktop with multiple resolution choices.

Now, let’s get the latest data and install. You may be asked to enter password for your user ID for Jetson Nano, and also if you want to continue. Type “y” and hit [Enter].

cd THEIA-CAM

chmod +x ./*.csh

./install.csh

Start Streaming

Now, your Raspberry Pi computer is ready to stream images. To start, connect one THSCU101 and try 1080p 30fps YUV.

./1080p30.csh

Voila! Do you see another window comes up showing live images? If the image is not clear, double check if you have removed the transparent protector on top of the 13MP PDAF sensor module. Ctrl+C to close the video streaming window.

You can stream different resolution, frame rate, and data format by selecting one of the command files as below.

./1080p30.csh

./1080p60.csh

./3M.csh

./4K.csh

./13M.csh

Now, connect the second THSCU101. To stream the secondary THEIA-CAM™, add “ /dev/video<n> to the video stream commands. In case of Raspberry Pi 400, the address should be /dev/video2. (The primal one is /dev/video0.) For the /dev/videos, your commands will be:

cd Desktop

cd THEIA-CAM

./1080p30.csh /dev/video2

./1080p60.csh /dev/video2

./3M.csh /dev/video2

./4K.csh /dev/video2

./13M.csh /dev/video2

By opening another Terminal window, you can see each video stream. To close video window, click Terminal window and Ctol+C.

What’s Next - Want to Tweak the Image Quality?

We are confident and proud of the Image Quality of the THEIA-CAM™ THSCU101. With Imatest criteria, the THSCU101 does perform better than iPhone 12 in most areas. However, different application requires different specs. If you need to change THEIA-CAM™’s configuration, there are several options listed as below.

UVC (USB Video Class) Configuration

Under years of close partnership with Cypress Semiconductor, an Infineon company, THSCU101 supports whole bunch of configuration set for UVC (USB Video Class) commands through Cypress’ CX3.

CDK (Camera Development Kit) tool to develop your own ISP firmware

From driving a CMOS image sensor, processing standard image such as Auto Exposure and Auto White Balance, and then Image Quality tuning in multiple different light conditions, to unit-to-unit Image Quality calibration against CMOS image sensor modules, an Image Signal Processor (ISP) covers everything that is related to image/video that Application Processor takes as the camera image output. Because the process, structure, and the firmware to execute all the computing, ISP’s process design is typically so closed and confidential that you cannot touch. THine’s ISP is the exception.

THine has developed a took set CDK (Camera Development Kit), that enables you to do the ISP firmware development for all the essential functions according to your own program requirements, with easy-to-use Graphical User Interface.

If CDK overview is found cover your program requirements, contact us and request for technical support. According to your application and requirements, we will tailor our support along with CDK took kit.

Related Pages

Product Overview | THSCU101 13MP PDAF UVC Camera | THine Solutions, Inc.

Image Quality Report | THSCU101 Image Quality Report | THine Solutions, Inc.

Whitepaper | Design Trend for Embedded Vision Systems that Include High Performance Cameras | Katsumi Kuwayama | THine Solutions, Inc.

Blog | Upgrade to PDAF for Best Auto-Focus Experience! | Tak Iizuka | THine Solutions, Inc.

Product Overview | THEIA-CAM - 13MP PDAF UVC Camera Reference Design Kit (RDK) | Cypress Semiconductor Corp.