Introduction to K230 AI Demo#
K230 AI Demo#
Overview#
The K230 AI Demo integrates modules such as face, human body, hand, license plate, word continuation, voice, DMS, etc., and includes various functions such as classification, detection, segmentation, recognition, tracking, and monocular distance measurement, providing customers with a reference on how to use K230 to develop AI-related applications.
Hardware Environment#
CanMV-K230-V1.x / CanMV-K230-V2.x / K230-UNSIP-LPx-EVB-V1.x
Source Code Location#
The source code path is located at k230_sdk/src/reference/ai_poc, with the directory structure as follows:
# Detailed demo instructions are available in the subdirectories of AI Demo (e.g., bytetrack, face_detection, etc.)
.
├── anomaly_det
├── build_app.sh
├── bytetrack
├── cmake
├── CMakeLists.txt
├── crosswalk_detect
├── dec_ai_enc
├── demo_mix
├── distraction_reminder
├── dms_system
├── dynamic_gesture
├── eye_gaze
├── face_alignment
├── face_detection
├── face_emotion
├── face_gender
├── face_glasses
├── face_landmark
├── face_mask
├── face_mesh
├── face_parse
├── face_pose
├── face_verification
├── falldown_detect
├── finger_guessing
├── fitness
├── head_detection
├── helmet_detect
├── kws
├── licence_det
├── licence_det_rec
├── llamac
├── nanotracker
├── object_detect_yolov8n
├── ocr
├── person_attr
├── person_detect
├── person_distance
├── pose_detect
├── pose_det_rtsp_plug
├── pphumanseg
├── puzzle_game
├── segment_yolov8n
├── self_learning
├── shell
├── smoke_detect
├── space_resize
├── sq_hand_det
├── sq_handkp_class
├── sq_handkp_det
├── sq_handkp_flower
├── sq_handkp_ocr
├── sq_handreco
├── traffic_light_detect
├── translate_en_ch
├── tts_zh
├── vehicle_attr
├── virtual_keyboard
└── yolop_lane_seg
The paths for kmodel, images, and related dependencies are located at k230_sdk/src/big/kmodel/ai_poc, with the directory structure as follows:
.
├── images
│ ├── 000.png
│ ├── 1000.jpg
│ ├── 1024x1111.jpg
│ ├── 1024x1331.jpg
│ ├── 1024x624.jpg
│ ├── 1024x768.jpg
│ ├── 333.jpg
│ ├── 640x340.jpg
│ ├── bus.jpg
│ ├── bytetrack_data
│ ├── car.jpg
│ ├── cw.jpg
│ ├── falldown_elder.jpg
│ ├── helmet.jpg
│ ├── hrnet_demo.jpg
│ ├── identification_card.png
│ ├── input_flower.jpg
│ ├── input_hd.jpg
│ ├── input_ocr.jpg
│ ├── input_pd.jpg
│ ├── licence.jpg
│ ├── person.png
│ ├── road.jpg
│ ├── smoke1.jpg
│ └── traffic.jpg
├── kmodel
│ ├── anomaly_det.kmodel
│ ├── bytetrack_yolov5n.kmodel
│ ├── cropped_test127.kmodel
│ ├── crosswalk.kmodel
│ ├── eye_gaze.kmodel
│ ├── face_alignment.kmodel
│ ├── face_alignment_post.kmodel
│ ├── face_detection_320.kmodel
│ ├── face_detection_640.kmodel
│ ├── face_detection_hwc.kmodel
│ ├── face_emotion.kmodel
│ ├── face_gender.kmodel
│ ├── face_glasses.kmodel
│ ├── face_landmark.kmodel
│ ├── face_mask.kmodel
│ ├── face_parse.kmodel
│ ├── face_pose.kmodel
│ ├── face_recognition.kmodel
│ ├── flower_rec.kmodel
│ ├── gesture.kmodel
│ ├── hand_det.kmodel
│ ├── handkp_det.kmodel
│ ├── hand_reco.kmodel
│ ├── head_detection.kmodel
│ ├── helmet.kmodel
│ ├── hifigan.kmodel
│ ├── human_seg_2023mar.kmodel
│ ├── kws.kmodel
│ ├── licence_reco.kmodel
│ ├── LPD_640.kmodel
│ ├── nanotrack_backbone_sim.kmodel
│ ├── nanotracker_head_calib_k230.kmodel
│ ├── ocr_det_int16.kmodel
│ ├── ocr_det.kmodel
│ ├── ocr_rec_int16.kmodel
│ ├── ocr_rec.kmodel
│ ├── person_attr_yolov5n.kmodel
│ ├── person_detect_yolov5n.kmodel
│ ├── person_pulc.kmodel
│ ├── recognition.kmodel
│ ├── traffic.kmodel
│ ├── translate_decoder.kmodel
│ ├── translate_encoder.kmodel
│ ├── vehicle_attr_yolov5n.kmodel
│ ├── vehicle.kmodel
│ ├── yolop.kmodel
│ ├── yolov5n-falldown.kmodel
│ ├── yolov5s_smoke_best.kmodel
│ ├── yolov8n_320.kmodel
│ ├── yolov8n_640.kmodel
│ ├── yolov8n-pose.kmodel
│ ├── yolov8n_seg_320.kmodel
│ ├── yolov8n_seg_640.kmodel
│ ├── zh_fastspeech_1_f32.kmodel
│ ├── zh_fastspeech_1.kmodel
│ └── zh_fastspeech_2.kmodel
└── utils
├── Asci0816.zf
├── bfm_tri.bin
├── bu.bin
├── dict_6625.txt
├── dict_ocr_16.txt
├── dict_ocr.txt
├── file
├── HZKf2424.hz
├── jiandao.bin
├── libsentencepiece.a
├── llama.bin
├── memory.bin
├── ncc_code.bin
├── pintu.bin
├── reply_wav
├── shang.bin
├── shitou.bin
├── tokenizer.bin
├── trans_src.model
├── trans_tag.model
├── wav_play.elf
├── xia.bin
├── you.bin
└── zuo.bin
Compiling and Running the Program#
a. Self-Compiling Board Image#
Ensure that you have constructed the Docker container and built the image according to the official K230 SDK instructions.
Navigate to k230_sdk/src/reference/ai_poc.
Execute the build_app.sh script, which will copy the kmodel, images, utils, and shell to the k230_bin folder.
Copy the entire k230_bin folder to the board and execute the sh script on the main core to run the corresponding AI demo.
(1) Ensure that you have constructed the Docker container and built the image according to the official K230 SDK instructions.
(2) Navigate to k230_sdk/src/reference/ai_poc.
cd k230_sdk/src/reference/ai_poc
(3) Compile a single AI Demo (e.g., face detection)
./build_app.sh face_detection
Note: Before executing build_app.sh, ensure that the dependencies are ready, i.e., the relevant kmodel, images, and utils are already present under k230_sdk/src/big/kmodel/ai_poc.
# If the dependencies are not present, execute the following command to download them
cd k230_sdk
make prepare_sourcecode
The following files will be generated:
k230_bin/
├── face_detection
│ ├── 1024x624.jpg
│ ├── face_detect_image.sh
│ ├── face_detection_320.kmodel
│ ├── face_detection_640.kmodel
│ ├── face_detection.elf
│ └── face_detect_isp.sh
Copy the entire k230_bin/ folder to the board, and execute the sh script on the main core to run the corresponding AI demo.
# Navigate to the sharefs directory of the small core on the development board
scp -r username@ip:/xxx/k230_sdk/src/big/kmodel/ai_poc/k230_bin .
# Navigate to the sharefs directory of the big core on the development board
cd /sharefs/k230_bin/face_detection
# Execute the corresponding script to run face detection
# Detailed face detection instructions can be found in k230_sdk/src/reference/ai_poc/face_detection/README.md
./face_detect_isp.sh
(4) Compile all AI Demos (if you only need to compile a specific demo, skip this step)
./build_app.sh
The following files will be generated:
k230_bin/
......
├── face_detection
│ ├── 1024x624.jpg
│ ├── face_detect_image.sh
│ ├── face_detection_320.kmodel
│ ├── face_detection_640.kmodel
│ ├── face_detection.elf
│ └── face_detect_isp.sh
......
└── llamac
├── llama.bin
├── llama_build.sh
├── llama_run
└── tokenizer.bin
......
(5) For the 01studio development board, which supports compiling AI demos for HDMI display and LCD display, the compilation commands are as follows:
./build_app.sh face_detection hdmi
./build_app.sh face_detection lcd
The above commands will generate compiled outputs for a single demo under different display modes. If you want to compile all AI demos, the compilation commands are as follows:
./build_app.sh all hdmi
./build_app.sh all lcd
The above commands will compile all AI demos according to the specified display mode. The compiled outputs for HDMI will be located in the k230_bin directory, while the compiled outputs for LCD display mode will be located in the k230_bin_lcd directory.
Copy the entire k230_bin/ folder to the board, and execute the sh script on the main core to run the corresponding AI demo.
# Navigate to the sharefs directory of the small core
scp -r username@ip:/xxx/k230_sdk/src/big/kmodel/ai_poc/k230_bin .
# Navigate to the sharefs directory of the big core
cd /sharefs/k230_bin/face_detection
# Execute the corresponding script to run face detection
# Detailed face detection instructions can be found in k230_sdk/src/reference/ai_poc/face_detection/README.md
./face_detect_isp.sh
b. Directly Download the Official Board Image#
(1) Ensure that you have constructed the Docker container according to the official K230 SDK instructions.
(2) Download the image from the official resource library.
(3) Environment Preparation
cd k230_sdk
make prepare_sourcecode # (ignore if already executed)
make mpp
make cdk-user
# Execute different commands based on the development board model
# If the model is CanMV-K230-V1.x, execute the following command
make CONF=k230_canmv_defconfig prepare_memory
# If the model is CanMV-K230-V2.x, execute the following command
make CONF=k230_canmv_v2_defconfig prepare_memory
# If the model is K230-xxx-xxx-EVB-V1.x, execute the following command
make CONF=k230_evb_defconfig prepare_memory
cd k230_sdk/src/reference/ai_poc
(4) Compile a Single AI Demo (e.g., face detection)
./build_app.sh face_detection
Note: Before executing build_app.sh, ensure that the dependencies are ready, i.e., the relevant kmodel, images, and utils are already present under k230_sdk/src/big/kmodel/ai_poc.
# If the dependencies are not present, execute the following command to download them
cd k230_sdk
make prepare_sourcecode
The following files will be generated:
k230_bin/
├── face_detection
│ ├── 1024x624.jpg
│ ├── face_detect_image.sh
│ ├── face_detection_320.kmodel
│ ├── face_detection_640.kmodel
│ ├── face_detection.elf
│ └── face_detect_isp.sh
Copy the entire k230_bin folder to the board, and execute the sh script on the big core to run the corresponding AI demo.
# Navigate to the sharefs directory of the small core on the development board
scp -r username@ip:/xxx/k230_sdk/src/big/kmodel/ai_poc/k230_bin .
# Navigate to the sharefs directory of the big core on the development board
cd /sharefs/k230_bin/face_detection
# Execute the corresponding script to run face detection
# Detailed face detection instructions can be found in k230_sdk/src/reference/ai_poc/face_detection/README.md
./face_detect_isp.sh
(5) Compile All AI Demos (if you only need to compile a specific demo, skip this step)
./build_app.sh
The following files will be generated:
k230_bin/
......
├── face_detection
│ ├── 1024x624.jpg
│ ├── face_detect_image.sh
│ ├── face_detection_320.kmodel
│ ├── face_detection_640.kmodel
│ ├── face_detection.elf
│ └── face_detect_isp.sh
......
└── llamac
├── llama.bin
├── llama_build.sh
├── llama_run
└── tokenizer.bin
......
Copy the entire k230_bin folder to the board, and execute the sh script on the big core to run the corresponding AI demo.
# Navigate to the sharefs directory of the small core
scp -r username@ip:/xxx/k230_sdk/src/big/kmodel/ai_poc/k230_bin .
# Navigate to the sharefs directory of the big core
cd /sharefs/k230_bin/face_detection
# Execute the corresponding script to run face detection
# Detailed face detection instructions can be found in k230_sdk/src/reference/ai_poc/face_detection/README.md
./face_detect_isp.sh