elchika elchika
ログイン/登録
編集履歴一覧に戻る
Satomiのアイコン画像

Satomi が 2021年05月15日23時47分00秒 に編集

初版

タイトルの変更

+

目覚ましマウスパッド

タグの変更

+

obnizBoard

+

赤外線受信モジュール

+

赤外線LED

+

DCモーター

+

Buzzer

+

face-api

本文の変更

+

# 目次 [1. 概要](#概要) [2. デモ動画](#デモ動画) # 概要 ブラウザ上でWebカメラを使用しFace-api.jsを用いて眠気検知を行う。 寝ていることを検知したらWifi経由でマウスパッドに搭載されたObnizに指令が送られ、目覚ましアクションが発生する。 アクションは眠気レベルによって変化する。 - レベル１　空調の温度が下がる - レベル２　マウスパッドが動いて起こしてくれる - レベル３　ブザーによる目覚まし - レベル４　ライトOFF（疲れてるから寝たほうがよい） ![キャプションを入力できます](https://camo.elchika.com/6fb7db168c001678ed90755e12bded7ad1bb1ef9/687474703a2f2f73746f726167652e676f6f676c65617069732e636f6d2f656c6368696b612f76312f757365722f30323366303566662d306139382d343961382d396365392d3432336564613332333061332f38393137646366362d373739642d346166622d396166342d376235353861623634353836/) ![マウスパッド全体像](https://camo.elchika.com/063a2e5d8384fdbb8589e18b6dba23a2ac4f362c/687474703a2f2f73746f726167652e676f6f676c65617069732e636f6d2f656c6368696b612f76312f757365722f30323366303566662d306139382d343961382d396365392d3432336564613332333061332f31333964616464392d393139662d346337662d616237392d396531353963373061636235/) # デモ動画 **眠気検知について** WebカメラとFace-api.jsで実現 [<< 動画 >>](https://youtu.be/gzsYHE1Xhwg) ![キャプションを入力できます](https://camo.elchika.com/2a8708befbfdde04397b63c1aa17fbe27bf1bae4/687474703a2f2f73746f726167652e676f6f676c65617069732e636f6d2f656c6368696b612f76312f757365722f30323366303566662d306139382d343961382d396365392d3432336564613332333061332f39396261323939392d333765372d343562662d386637322d616139383035346635303463/) 眠気検知方法はFace-api.jsを用いて目の特徴点を抽出 【参考】<https://github.com/justadudewhohacks/face-api.js/> 目の開閉判定は以下記事を参考にした。 【参考】<https://www.pyimagesearch.com/2017/04/24/eye-blink-detection-opencv-python-dlib/> 閉じている時間から、眠っているか否かの判定をする。 眠っていると判定された回数が多くなるにつれ、眠気レベルを上げていく。 **空調、電気の操作について** Obnizに赤外線受信モジュールと、赤外線LEDを接続。 ブラウザ上の登録画面から各信号を登録。 ![obniz：赤外線受信モジュール、赤外線LED、ブザー](https://camo.elchika.com/97fc94756adcdfeec8bafa8ae992c03969101207/687474703a2f2f73746f726167652e676f6f676c65617069732e636f6d2f656c6368696b612f76312f757365722f30323366303566662d306139382d343961382d396365392d3432336564613332333061332f36316439343836662d613035662d346134352d613965322d306466323038336463663561/) ![赤外線データ登録画面](https://camo.elchika.com/96f387c4c8b503ff363e320f1d2aa9b45bed2c40/687474703a2f2f73746f726167652e676f6f676c65617069732e636f6d2f656c6368696b612f76312f757365722f30323366303566662d306139382d343961382d396365392d3432336564613332333061332f39613932346439322d383639632d343462642d623835312d393666366632316162393438/) **振動機構について** DCモーターとレゴブロックで実現 動画：https://youtu.be/2pcjLCafXPY ![LEGO 振動機構](https://camo.elchika.com/b5143742ba04e22813e2c3cfcb1d3ff6fa636169/687474703a2f2f73746f726167652e676f6f676c65617069732e636f6d2f656c6368696b612f76312f757365722f30323366303566662d306139382d343961382d396365392d3432336564613332333061332f37613230633832382d356166632d346133382d626439392d316163396638366233316535/) ```arduino:Main(GUI+眠気検知+Obniz操作) <html> <head> <meta charset="utf-8" /> <meta name="viewport" content="width=device-width, initial-scale=1" /> <link rel="stylesheet" href="https://stackpath.bootstrapcdn.com/bootstrap/4.3.1/css/bootstrap.min.css" /> <script src="https://code.jquery.com/jquery-3.2.1.min.js"></script> <script src="https://unpkg.com/obniz@3.x/obniz.js" crossorigin="anonymous" ></script> <script src="https://unpkg.com/obniz-parts-kits@0.16.0/ui/index.js"></script> <script src="https://obniz.com/users/5143/repo/storage.json"></script> <script src="https://obniz.com/users/5143/repo/register_data.js"></script> <script src="https://obniz.com/users/5143/repo/face-api.min.js"></script> </head> <body> <!-- webカメラ画面表示 --> <div style="position: relative"> <video id="video" onloadedmetadata="onPlay(this)" muted autoplay width="640" height="480"></video> <canvas id="facecanvas" style=" position: absolute; top: 0; left: 0;" width="640" height="480"></canvas> </div> <div id="obniz-debug"></div> <div class="container"> <div class="text-center"> <h3>Sleepiness Detection</h3> <label>電源 ON : </label> <button class="btn btn-link" id="power_on_reg">確認中</button> <button class="btn btn-info" id="power_on_send">実行</button> <label id="power_on_sts"></label> <br> <label>電源 OFF : </label> <button class="btn btn-link" id="power_off_reg">確認中</button> <button class="btn btn-info" id="power_off_send">実行</button> <label id="power_off_sts"></label> <br> <label>ライト ON : </label> <button class="btn btn-link" id="light_on_reg">確認中</button> <button class="btn btn-info" id="light_on_send">実行</button> <label id="light_on_sts"></label> <br> <label>ライト OFF : </label> <button class="btn btn-link" id="light_off_reg">確認中</button> <button class="btn btn-info" id="light_off_send">実行</button> <label id="light_off_sts"></label> <br> <label>エアコン ON : </label> <button class="btn btn-link" id="aircon_on_reg">確認中</button> <button class="btn btn-info" id="aircon_on_send">実行</button> <label id="aircon_on_sts"></label> <br> <label>エアコン OFF : </label> <button class="btn btn-link" id="aircon_off_reg">確認中</button> <button class="btn btn-info" id="aircon_off_send">実行</button> <label id="aircon_off_sts"></label> <br> <label>温度 上げる : </label> <button class="btn btn-link" id="thm_up_reg">確認中</button> <button class="btn btn-info" id="thm_up_send">実行</button> <label id="thm_up_sts"></label> <br> <label>温度 下げる : </label> <button class="btn btn-link" id="thm_down_reg">確認中</button> <button class="btn btn-info" id="thm_down_send">実行</button> <label id="thm_down_sts"></label> <br> </div> </div> <script> var obniz = new Obniz("OBNIZ_ID_HERE"); var ObnizId = document.getElementById('ObnizId'); var power_on_reg = document.getElementById('power_on_reg'); var power_on_send = document.getElementById('power_on_send'); var power_on_sts = document.getElementById('power_on_sts'); var power_off_reg = document.getElementById('power_off_reg'); var power_off_send = document.getElementById('power_off_send'); var power_off_sts = document.getElementById('power_off_sts'); var light_on_reg = document.getElementById('light_on_reg'); var light_on_send = document.getElementById('light_on_send'); var light_on_sts = document.getElementById('light_on_sts'); var light_off_reg = document.getElementById('light_off_reg'); var light_off_send = document.getElementById('light_off_send'); var light_off_sts = document.getElementById('light_off_sts'); var aircon_on_reg = document.getElementById('aircon_on_reg'); var aircon_on_send = document.getElementById('aircon_on_send'); var aircon_on_sts = document.getElementById('aircon_on_sts'); var aircon_off_reg = document.getElementById('aircon_off_reg'); var aircon_off_send = document.getElementById('aircon_off_send'); var aircon_off_sts = document.getElementById('aircon_off_sts'); var thm_up_reg = document.getElementById('thm_up_reg'); var thm_up_send = document.getElementById('thm_up_send'); var thm_up_sts = document.getElementById('thm_up_sts'); var thm_down_reg = document.getElementById('thm_down_reg'); var thm_down_send = document.getElementById('thm_down_send'); var thm_down_sts = document.getElementById('thm_down_sts'); var res_promise; const canvas = document.getElementById( 'facecanvas' ); const videoEl = document.getElementById( 'video' ); const ctx = canvas.getContext("2d"); ctx.font = "32px serif"; ctx.fillStyle = "Red"; const inputSize = 224; const scoreThreshold = 0.5; const options = new faceapi.TinyFaceDetectorOptions({ inputSize, scoreThreshold }); var PassSec = 0;   // 秒数カウント用変数 var WakeupCount = 0;   // 秒数カウント用変数 var flg = true; var sleepinessCount = 0; var sleepiness_level = 0; var power_off_flag = false; var thm_down_flag = false; var PassSeccount_flag = false; timerID = setInterval('countup()',1000); //1秒毎にcountup()を呼び出し const xml = new XMLHttpRequest(); xml.open("POST", "https://maker.ifttt.com/trigger/WakeupAlarm/with/key/d5nIKdxHkXiL3bjtNe6_t_", false); xml.setRequestHeader("content-type", "application/x-www-form-urlencoded;charset=UTF-8","Access-Control-Allow-Origin", "https://maker.ifttt.com/trigger/WakeupAlarm/with/key/d5nIKdxHkXiL3bjtNe6_t_"); function countup() { PassSec++; PassSeccount_flag = true; } async function onPlay() { if(videoEl.paused || videoEl.ended || !faceapi.nets.tinyFaceDetector.params) return setTimeout(() => onPlay()) // face-apiを使って, 両目の特徴点を取得 const result = await faceapi.detectSingleFace(videoEl, options).withFaceLandmarks() if (result) { const dims = faceapi.matchDimensions(canvas, videoEl, true) const resizedResult = faceapi.resizeResults(result, dims) const leftEye = result.landmarks.getLeftEye() const rightEye = result.landmarks.getRightEye() // EAR(Eye Aspect Ratio)を算出 var a_l = Math.sqrt( Math.pow( leftEye[1].x-leftEye[5].x, 2 ) + Math.pow( leftEye[1].y-leftEye[5].y, 2 ) ) ; var b_l = Math.sqrt( Math.pow( leftEye[2].x-leftEye[4].x, 2 ) + Math.pow( leftEye[2].y-leftEye[4].y, 2 ) ) ; var c_l = Math.sqrt( Math.pow( leftEye[0].x-leftEye[3].x, 2 ) + Math.pow( leftEye[0].y-leftEye[3].y, 2 ) ) ; var EAR_L = ( a_l + b_l ) / ( 2 * c_l ) ; var a_r = Math.sqrt( Math.pow( rightEye[1].x-rightEye[5].x, 2 ) + Math.pow( rightEye[1].y-rightEye[5].y, 2 ) ) ; var b_r = Math.sqrt( Math.pow( rightEye[2].x-rightEye[4].x, 2 ) + Math.pow( rightEye[2].y-rightEye[4].y, 2 ) ) ; var c_r = Math.sqrt( Math.pow( rightEye[0].x-rightEye[3].x, 2 ) + Math.pow( rightEye[0].y-rightEye[3].y, 2 ) ) ; var EAR_R = ( a_r + b_r ) / ( 2 * c_r ) ; var EAR = ( EAR_R + EAR_L ) / 2; ctx.fillText("眠気検知回数：" + sleepinessCount, 20, 20); ctx.fillText("レベル：" + sleepiness_level, 20, 40); ctx.fillText(flg, 20, 100); //目があいているフレーム数をカウント if(EAR > 0.33){ // 使用者によりチューニングが必要!! WakeupCount++; } //目があいているフレーム数をが5フレーム以上は起きている if(WakeupCount > 5){ PassSec = 0; WakeupCount = 0; } if(PassSec == 5){ WakeupCount = 0; if(PassSeccount_flag == true){ sleepinessCount ++ ; if(sleepinessCount < 2){ sleepiness_level = 1 ; }else if(sleepinessCount < 4){ sleepiness_level = 2 ; }else if(sleepinessCount < 6){ sleepiness_level = 3 ; }else if(sleepinessCount < 7){ sleepiness_level = 4 ; }else{ sleepiness_level = 5 ; } } ctx.fillText("SLEEP", 20, 120); flg = false; }else if(PassSec > 5){ ctx.fillText("SLEEP", 20, 120); }else{ ctx.fillText("AWAKE", 20, 120); flg = true; } } PassSeccount_flag = false; setTimeout(() => onPlay()) }; async function run(){ await faceapi.nets.tinyFaceDetector.load('https://obniz.com/users/5143/repo/') await faceapi.loadFaceLandmarkModel('https://obniz.com/users/5143/repo/') const stream = await navigator.mediaDevices.getUserMedia({ video: {} }) videoEl.srcObject = stream; } $(document).ready(function() { run(); }); // called on online obniz.onconnect = async function() { var sensor = obniz.wired('IRSensor', {vcc:2, gnd:1, output: 0}); var led = obniz.wired('InfraredLED', {anode: 4, cathode: 3}); var speaker = obniz.wired("Speaker", {signal:5 , gnd:6 }); var dcmotor = obniz.wired("DCMotor",{forward:10, back:11}); var power_on = []; var power_off = []; var light_on = []; var light_off = []; // Load console.log('Load Start'); var power_on_reg_data = new Register(obniz, "PowerOn" , sensor, led, power_on_reg , power_on_sts); var power_off_reg_data = new Register(obniz, "PowerOff" , sensor, led, power_off_reg , power_off_sts); var light_on_reg_data = new Register(obniz, "light_on" , sensor, led, light_on_reg , light_on_sts); var light_off_reg_data = new Register(obniz, "light_off" , sensor, led, light_off_reg , light_off_sts); var aircon_on_reg_data = new Register(obniz, "aircon_on" , sensor, led, aircon_on_reg , aircon_on_sts); var aircon_off_reg_data = new Register(obniz, "aircon_off" , sensor, led, aircon_off_reg , aircon_off_sts); var thm_up_reg_data = new Register(obniz, "thm_up" , sensor, led, thm_up_reg , thm_up_sts); var thm_down_reg_data = new Register(obniz, "thm_down" , sensor, led, thm_down_reg , thm_down_sts); power_on_reg_data.LoadData(); power_off_reg_data.LoadData(); light_on_reg_data.LoadData(); light_off_reg_data.LoadData(); aircon_on_reg_data.LoadData(); aircon_off_reg_data.LoadData(); thm_up_reg_data.LoadData(); thm_down_reg_data.LoadData(); $("#power_on_reg").click(function() { power_on_reg_data.RegisterData(); sensor.ondetect = function(arr) { power_on_reg_data.RegisterComp(arr); } }); $("#power_off_reg").click(function() { power_off_reg_data.RegisterData(); sensor.ondetect = function(arr) { power_off_reg_data.RegisterComp(arr); } }); $("#light_on_reg").click(function() { light_on_reg_data.RegisterData(); sensor.ondetect = function(arr) { light_on_reg_data.RegisterComp(arr); } }); $("#light_off_reg").click(function() { light_off_reg_data.RegisterData(); sensor.ondetect = function(arr) { light_off_reg_data.RegisterComp(arr); } }); $("#aircon_on_reg").click(function() { aircon_on_reg_data.RegisterData(); sensor.ondetect = function(arr) { aircon_on_reg_data.RegisterComp(arr); } }); $("#aircon_off_reg").click(function() { aircon_off_reg_data.RegisterData(); sensor.ondetect = function(arr) { aircon_off_reg_data.RegisterComp(arr); } }); $("#thm_up_reg").click(function() { thm_up_reg_data.RegisterData(); sensor.ondetect = function(arr) { thm_up_reg_data.RegisterComp(arr); } }); $("#thm_down_reg").click(function() { thm_down_reg_data.RegisterData(); sensor.ondetect = function(arr) { thm_down_reg_data.RegisterComp(arr); } }); // Send $("#power_on_send").click(async function() { power_on_reg_data.SendData(); }); $("#power_off_send").click(async function() { power_off_reg_data.SendData(); }); $("#light_on_send").click(async function() { light_on_reg_data.SendData(); }); $("#light_off_send").click(async function() { light_off_reg_data.SendData(); }); $("#aircon_on_send").click(async function() { aircon_on_reg_data.SendData(); }); $("#aircon_off_send").click(async function() { aircon_off_reg_data.SendData(); }); $("#thm_up_send").click(async function() { thm_up_reg_data.SendData(); }); $("#thm_down_send").click(async function() { thm_down_reg_data.SendData(); }); // called while online. obniz.onloop = async function() { const _sleep = (ms) => new Promise((resolve) => setTimeout(resolve, ms)); switch (sleepiness_level){ case 1: //空調の温度を下げる if(flg == false){ if( thm_down_flag == false){ aircon_on_reg_data.SendData(); await _sleep(500); thm_down_reg_data.SendData(); thm_down_flag = true; } } break; case 2: //マウスパットを振動する if(flg == false){ dcmotor.power(70); for (let i = 0; i < 3; i++) { dcmotor.move(false); await _sleep(300); dcmotor.stop(); await _sleep(250); dcmotor.move(true); await _sleep(300); dcmotor.stop(); await _sleep(250); } } break; case 3:　//ブザーを鳴らす if(flg == false){ speaker.play(1000); } else{ speaker.stop(); } break; case 4:　//ライトを消す speaker.stop(); if( power_off_flag == false){ power_off_reg_data.SendData(); power_off_flag = true; } break; default: break; } }; }; // called on offline obniz.onclose = async function() { }; </script> </body> </html> ``` ```arduino:赤外線データ登録 var Register = function(obniz, key, sensor, led, reg_status, load_status) { this.obniz = obniz; this.key = key; this.sensor = sensor; this.led = led; this.reg_status = reg_status; this.load_status = load_status; this.data = []; this.LoadData = async function(){ var res_promise; res_promise = (ObnizUI.Util.loadFromStorage(this.key)); res_promise.then( response => { if (Array.isArray(response)){ this.reg_status.innerText = "登録済み"; this.data = response.concat(); } else{ this.reg_status.innerText = "未登録"; } }, error => { this.reg_status.innerText = "読み出し失敗"; } ); return 0; } this.RegisterData = async function(){ var res_data = []; if(this.reg_status.innerText == "登録中") { this.LoadData (); } else { this.reg_status.innerText = "登録中"; this.obniz.display.print("detecting..."); this.sensor.start(); } } this.RegisterComp = async function(arr){ var res_data = []; if (this.reg_status.innerText == "登録中"){ this.reg_status.innerText = "登録完了"; this.obniz.display.print("detecte success"); res_data = arr.concat(); ObnizUI.Util.saveToStorage(this.key, res_data); this.LoadData (); } } this.SendData = function(){ if ((this.reg_status.innerText == "登録済み") || (this.reg_status.innerText == "登録完了")) { this.led.send(this.data); } else { alert("未登録です"); } } }; ```