Airflow FileSensor taks 不工作,文件路径出来很奇怪
python 230
原文标题 :Airflow FileSensor taks is not working,The file path is coming out weird
Airflow FileSensor taks 不工作,一直处于运行状态。我创建了一个文件连接,如下所示。
连接 ID:file_sensor 连接类型:文件(路径)额外:{“路径”:“/opt/airflow/sensor”}
我不知道为什么 AIRFLOW_CTX_DAG_OWNER 显示为 ****。
*** Reading local file: /opt/airflow/logs/sensor_test/sensor_a/2022-03-24T07:52:07.069026+00:00/1.log
[2022-03-24, 16:52:08 KST] {taskinstance.py:1037} INFO - Dependencies all met for <TaskInstance: sensor_test.sensor_a manual__2022-03-24T07:52:07.069026+00:00 [queued]>
[2022-03-24, 16:52:08 KST] {taskinstance.py:1037} INFO - Dependencies all met for <TaskInstance: sensor_test.sensor_a manual__2022-03-24T07:52:07.069026+00:00 [queued]>
[2022-03-24, 16:52:08 KST] {taskinstance.py:1243} INFO -
--------------------------------------------------------------------------------
[2022-03-24, 16:52:08 KST] {taskinstance.py:1244} INFO - Starting attempt 1 of 1
[2022-03-24, 16:52:08 KST] {taskinstance.py:1245} INFO -
--------------------------------------------------------------------------------
[2022-03-24, 16:52:08 KST] {taskinstance.py:1264} INFO - Executing <Task(FileSensor): sensor_a> on 2022-03-24 07:52:07.069026+00:00
[2022-03-24, 16:52:08 KST] {standard_task_runner.py:52} INFO - Started process 960 to run task
[2022-03-24, 16:52:08 KST] {standard_task_runner.py:76} INFO - Running: ['***', 'tasks', 'run', 'sensor_test', 'sensor_a', 'manual__2022-03-24T07:52:07.069026+00:00', '--job-id', '349', '--raw', '--subdir', 'DAGS_FOLDER/***SensorTest.py', '--cfg-path', '/tmp/tmp5wiingk3', '--error-file', '/tmp/tmplhcg37hy']
[2022-03-24, 16:52:08 KST] {standard_task_runner.py:77} INFO - Job 349: Subtask sensor_a
[2022-03-24, 16:52:08 KST] {logging_mixin.py:109} INFO - Running <TaskInstance: sensor_test.sensor_a manual__2022-03-24T07:52:07.069026+00:00 [running]> on host b05251258af1
[2022-03-24, 16:52:08 KST] {taskinstance.py:1431} INFO - Exporting the following env vars:
AIRFLOW_CTX_DAG_OWNER=***
AIRFLOW_CTX_DAG_ID=sensor_test
AIRFLOW_CTX_TASK_ID=sensor_a
AIRFLOW_CTX_EXECUTION_DATE=2022-03-24T07:52:07.069026+00:00
AIRFLOW_CTX_DAG_RUN_ID=manual__2022-03-24T07:52:07.069026+00:00
[2022-03-24, 16:52:08 KST] {filesystem.py:59} INFO - Poking for file /opt/***/sensor/a.txt
[2022-03-24, 16:53:08 KST] {filesystem.py:59} INFO - Poking for file /opt/***/sensor/a.txt
[2022-03-24, 16:54:08 KST] {filesystem.py:59} INFO - Poking for file /opt/***/sensor/a.txt
[2022-03-24, 16:55:08 KST] {filesystem.py:59} INFO - Poking for file /opt/***/sensor/a.txt
[2022-03-24, 16:56:08 KST] {filesystem.py:59} INFO - Poking for file /opt/***/sensor/a.txt
[2022-03-24, 16:57:08 KST] {filesystem.py:59} INFO - Poking for file /opt/***/sensor/a.txt
[2022-03-24, 16:58:08 KST] {filesystem.py:59} INFO - Poking for file /opt/***/sensor/a.txt
[2022-03-24, 16:59:08 KST] {filesystem.py:59} INFO - Poking for file /opt/***/sensor/a.txt
[2022-03-24, 17:00:08 KST] {filesystem.py:59} INFO - Poking for file /opt/***/sensor/a.txt
[2022-03-24, 17:01:08 KST] {filesystem.py:59} INFO - Poking for file /opt/***/sensor/a.txt
[2022-03-24, 17:02:08 KST] {filesystem.py:59} INFO - Poking for file /opt/***/sensor/a.txt
[2022-03-24, 17:03:08 KST] {filesystem.py:59} INFO - Poking for file /opt/***/sensor/a.txt
[2022-03-24, 17:04:08 KST] {filesystem.py:59} INFO - Poking for file /opt/***/sensor/a.txt
[2022-03-24, 17:05:09 KST] {filesystem.py:59} INFO - Poking for file /opt/***/sensor/a.txt
[2022-03-24, 17:06:09 KST] {filesystem.py:59} INFO - Poking for file /opt/***/sensor/a.txt
[2022-03-24, 17:07:09 KST] {filesystem.py:59} INFO - Poking for file /opt/***/sensor/a.txt
[2022-03-24, 17:08:09 KST] {filesystem.py:59} INFO - Poking for file /opt/***/sensor/a.txt
[2022-03-24, 17:09:09 KST] {filesystem.py:59} INFO - Poking for file /opt/***/sensor/a.txt
[2022-03-24, 17:10:09 KST] {filesystem.py:59} INFO - Poking for file /opt/***/sensor/a.txt
[2022-03-24, 17:11:09 KST] {filesystem.py:59} INFO - Poking for file /opt/***/sensor/a.txt
[2022-03-24, 17:12:09 KST] {filesystem.py:59} INFO - Poking for file /opt/***/sensor/a.txt
[2022-03-24, 17:13:09 KST] {filesystem.py:59} INFO - Poking for file /opt/***/sensor/a.txt
[2022-03-24, 17:14:09 KST] {filesystem.py:59} INFO - Poking for file /opt/***/sensor/a.txt
[2022-03-24, 17:15:09 KST] {filesystem.py:59} INFO - Poking for file /opt/***/sensor/a.txt
[2022-03-24, 17:16:09 KST] {filesystem.py:59} INFO - Poking for file /opt/***/sensor/a.txt
[2022-03-24, 17:17:09 KST] {filesystem.py:59} INFO - Poking for file /opt/***/sensor/a.txt
[2022-03-24, 17:18:09 KST] {filesystem.py:59} INFO - Poking for file /opt/***/sensor/a.txt
[2022-03-24, 17:19:09 KST] {filesystem.py:59} INFO - Poking for file /opt/***/sensor/a.txt
[2022-03-24, 17:20:09 KST] {filesystem.py:59} INFO - Poking for file /opt/***/sensor/a.txt
[2022-03-24, 17:21:10 KST] {filesystem.py:59} INFO - Poking for file /opt/***/sensor/a.txt
[2022-03-24, 17:22:10 KST] {filesystem.py:59} INFO - Poking for file /opt/***/sensor/a.txt
[2022-03-24, 17:23:10 KST] {filesystem.py:59} INFO - Poking for file /opt/***/sensor/a.txt
[2022-03-24, 17:24:10 KST] {filesystem.py:59} INFO - Poking for file /opt/***/sensor/a.txt
[2022-03-24, 17:25:10 KST] {filesystem.py:59} INFO - Poking for file /opt/***/sensor/a.txt