Teams会议侧边栏应用开发-会议转写

Teams应用开发，主要是权限比较麻烦，大量阅读和实践，摸索了几周，才搞明白。现将经验总结如下：

一、目标：开发一个Teams会议的侧边栏应用，实现会议的实时转写。

二、前提：

1）Teams 365基础版本以上账号Developer Portal，主要是可以登录开发者门户，建议开放管理员权限，以便可以上传开发好的APP（实际上仅仅是个mainfest.json）；

2）Teams 基础版本以上账号，可以登录Teams，添加App；

3）Azure 账号，建议开放管理员权限，以方便授予同意权限。

以上是基本要求，否则无法进行后续的工作。

三、需求分解：

1）侧边栏->关键配置（configurableTabs）

2）实时转写->转写->会议组织者ID->会议ID->用户ID

四、涉及的权限：

1）用户token=委托权限(Delegated)，如：转录，需要有会议组织者ID的用户权限；

2）应用token=应用权限(Application)，如：获取转录列表。

以上两类权限，可以通过在Azure注册应用获得，如：

五、辅助工具：

1）Graph Explorer | Try Microsoft Graph APIs - Microsoft Graph可以帮助你调试API，判断是否有权限，及需要什么权限，如：User.Read, OnlineMeetingTranscript.Read.All 等。

2）jwt.ms: Welcome!帮助你判断获得的Token是哪类Token， typ=user or app。

基本要求交代完成后，下面说说具体的，正式开始。

六、注册Azure 应用

1）先找到入口-应用注册

2）记录注册应用ID和租户ID（获取user token或 app token都要用到）

3）增加一个密钥并记录，后面就看不到了。

4）授予权限，非常重要，否则无法调用对应到接口。

5）需要设置一个回调地址（user token需要）

6）隐式获取设置（可以一步直接获取user token，存储在回调页面/auth的heders URL hash片段）

正常应该是先获取code，然后拿code换token。

以上关于注册应用的设置全部完毕。

七、应用开发

1）开发工具使用VS Code，下载Teams Tookit插件，创建一个Tab应用，使用JS语言，应用名称随意，如：MeetingRTT。

2）mainfest.json，这个很关键。注意3个地方：id(注册应用ID)、configurableTabs(侧边栏配置)和validDomains（合法域名）。

{"$schema": "https://developer.microsoft.com/json-schemas/teams/v1.16/MicrosoftTeams.schema.json","manifestVersion": "1.16","version": "1.0.0","id": "ba82be1b-xxx","packageName": "com.helport.transcription","developer": {"name": "Helport","websiteUrl": "https://your-website.com","privacyUrl": "https://your-website.com/privacy","termsOfUseUrl": "https://your-website.com/terms"},"name": {"short": "Meeting Transcription","full": "Real-time Meeting Transcription"},"description": {"short": "Real-time meeting transcription","full": "This app provides real-time meeting transcription in Teams meetings."},"icons": {"outline": "outline.png","color": "color.png"},"accentColor": "#FFFFFF","configurableTabs": [{"configurationUrl": "https://xxx.ngrok-free.app/config","canUpdateConfiguration": true,"scopes": ["team","groupchat"],"context": ["meetingSidePanel","meetingStage"]}],"permissions": ["identity","messageTeamMembers"],"validDomains": ["xxx.ngrok-free.app"]
}

3) 侧边栏安装配置：一个html页面config.html

<!DOCTYPE html>
<html>
<head><title>Configure Transcription</title><script src="https://statics.teams.cdn.office.net/sdk/v1.11.0/js/MicrosoftTeams.min.js"></script>
</head>
<body><button id="save">Save</button><script>microsoftTeams.initialize();document.getElementById('save').addEventListener('click', () => {microsoftTeams.settings.setSettings({entityId: "transcriptionPanel",contentUrl: "https://xxx.ngrok-free.app/meetingTab",suggestedDisplayName: "Helport"});microsoftTeams.settings.setValidityState(true);microsoftTeams.settings.registerOnSaveHandler((saveEvent) => {saveEvent.notifySuccess();});});</script>
</body>
</html>

4) 本地服务端点实现，主要是为了解决跨域访问问题，需要https://，可以使用ngrok弄个免费的。

本地服务端点，主要实现有：

/confg，上面的侧边栏安装配置页面；

/meetingTab，侧边栏主页面，完成转写；

/auth，完成user token的获取，返回一个页面获取access_token(user_token)，存储在redis中；

/store_user_token，存储access_token(user_tokne)到redis中；

/get_user_token，从redis获取acces_token（user_token)，页面获取会议信息需要；

/getTranscripts，获取转录列表，需要使用app token；

/getTranscriptContent，获取转写，需要使用user_token。

import restify from "restify";
import send from "send";
import fs from "fs";
import fetch from "node-fetch";
import path from 'path';
import { fileURLToPath } from 'url';
import { storeToken, getToken } from './redisClient.js';const __filename = fileURLToPath(import.meta.url);
console.log('__filename: ', __filename);const __dirname = path.dirname(__filename);
console.log('__dirname: ', __dirname);// Create HTTP server.
const server = restify.createServer({key: process.env.SSL_KEY_FILE ? fs.readFileSync(process.env.SSL_KEY_FILE) : undefined,certificate: process.env.SSL_CRT_FILE ? fs.readFileSync(process.env.SSL_CRT_FILE) : undefined,formatters: {"text/html": function (req, res, body) {return body;},},
});server.use(restify.plugins.bodyParser());
server.use(restify.plugins.queryParser());server.get("/static/*",restify.plugins.serveStatic({directory: __dirname,})
);server.listen(process.env.port || process.env.PORT || 3000, function () {console.log(`\n${server.name} listening to ${server.url}`);
});// Adding tabs to our app. This will setup routes to various views
// Setup home page
server.get("/config", (req, res, next) => {send(req, __dirname + "/config/config.html").pipe(res);
});// Setup the static tab
server.get("/meetingTab", (req, res, next) => {send(req, __dirname + "/panel/panel.html").pipe(res);
});//获得用户token
server.get('/auth', (req, res, next) => {res.status(200);res.send(`
<!DOCTYPE html>
<html>
<head><script>// Function to handle the token storageasync function handleToken() {const hash = window.location.hash.substring(1);const hashParams = new URLSearchParams(hash);const access_token = hashParams.get('access_token');console.log('Received hash parameters:', hashParams);if (access_token) {console.log('Access token found:', access_token);localStorage.setItem("access_token", access_token);console.log('Access token stored in localStorage');try {const response = await fetch('https://shortly-adapted-akita.ngrok-free.app/store_user_token', {method: 'POST',headers: {'Content-Type': 'application/json'},body: JSON.stringify({ "user_token" : access_token })});if (response.ok) {console.log('Token stored successfully');} else {console.error('Failed to store token:', response.statusText);}} catch (error) {console.error('Error storing token:', error);}} else {console.log('No access token found');}window.close();}// Call the function to handle the tokenhandleToken();</script>
</head>
<body></body>
</html>`);next();
});// 存储 user_token
server.post('/store_user_token', async (req, res) => {const user_token = req.body.user_token;if (!user_token) {res.status(400);res.send('user_token are required');}try {// Store user tokenawait storeToken('user_token', user_token);console.log('user_token stored in Redis');} catch (err) {console.error('user_token store Error:', err);}res.status(200);   res.send('Token stored successfully');
});// 获取 user_token
server.get('/get_user_token', async (req, res) => {try {// Store user tokenconst user_token = await getToken('user_token');console.log('user_token get in Redis');res.send({"user_token": user_token});} catch (err) {console.error('user_token get Error:', err);}
});//应用token
let app_token = '';// 定义 /getTranscripts 端点
server.get('/getTranscripts', async (req, res) => {try {let token = "";if (app_token !=''){token = app_token}else{// 构建请求体const requestBody = new URLSearchParams({"grant_type": "client_credentials","client_id": client_id, //注册应用ID"client_secret": client_secret, //主要应用密钥"scope": "https://graph.microsoft.com/.default", //默认范围，即注册应用配置的所有权限}).toString();// 获取app令牌const tokenUrl = `https://login.microsoftonline.com/${tenant_id}/oauth2/v2.0/token`;const tokenResponse = await fetch(tokenUrl, {method: 'POST',headers: {'Content-Type': 'application/x-www-form-urlencoded',},body: requestBody,});if (!tokenResponse.ok) {const errorData = await tokenResponse.json();res.send(500, { error: errorData.error_description });}const tokenData = await tokenResponse.json();app_token = tokenData.access_tokentoken = app_tokenconsole.log("app_token recevied!")}const organizerId = req.query.organizerId;if (!organizerId) {res.send(400, { error: 'Organizer ID is required' });}// 调用 Microsoft Graph APIconst graphUrl = `https://graph.microsoft.com/beta/users/${organizerId}/onlineMeetings/getAllTranscripts(meetingOrganizerUserId='${organizerId}')/delta`;const graphResponse = await fetch(graphUrl, {headers: {Authorization: `Bearer ${token}`,},});if (!graphResponse.ok) {const errorData = await graphResponse.json();res.send(500, { error: errorData.error.message });}const data = await graphResponse.json();// 返回转录文本res.send(200, data);} catch (error) {// 返回错误res.send(500, { error: error.message });}
});// 定义 /getTranscriptContent 端点
server.get('/getTranscriptContent', async (req, res) => {try {const response = await fetch('https://xxx.ngrok-free.app/get_user_token');const token_data = await response.json();const transcriptContentUrl = req.query.transcriptContentUrl;if (!transcriptContentUrl) {res.send(400, { error: 'transcriptContentUrl is required' });}const content_url = `${transcriptContentUrl}?$format=text/vtt`;console.log('content_url:', content_url)// 调用 Microsoft Graph APIconst graphResponse = await fetch(content_url, {headers: {Authorization: `Bearer ${token_data.user_token}`,},});if (!graphResponse.ok) {const errorData = await graphResponse.text();res.send(500, { error: errorData.error.message });}const data = await graphResponse.text();console.log('data:', data)// 返回转录文本res.send(200, data);} catch (error) {// 返回错误res.send(500, { error: error.message });}
});

5）侧边栏页面

获取当前登录用户的信息，获取会议信息，获取会议组织者ID，获取转录列表，获取转写。前提是需要获取user token才可以获取用户信息，这里使用microsoftTeams.authentication.authenticate的url去打开一个授权登录页面，由于当前用户已经登录了，默认不需要去登录，会重定向到/auth，并将user token推送到/auth页面，该/auth服务端点接收到后，存储在redis中，然后关闭页面，之所以使用redis是因为其它存储都无效，从定向后，localStorage、sessionStorage都无法保存，该页面在浏览器中和在teams中完全是两个环境，这个两个环境的数据无法交换，所以只能使用redis在服务端存储。

<!DOCTYPE html>
<html lang="en">
<head><meta charset="UTF-8"><meta name="viewport" content="width=device-width, initial-scale=1.0"><title>Teams User Info</title><script src="https://res.cdn.office.net/teams-js/2.0.0/js/MicrosoftTeams.min.js"></script>
</head>
<body><button id="fetchTranscripts">Fetch Transcripts</button><h2>Meeting Transcripts</h2><div id="transcripts"></div><script>const clientId ='your_client_id';const tenantId = 'your_tentant_id';const authUrl = `https://login.microsoftonline.com/${tenantId}/oauth2/v2.0/authorize`;const redirectUri = 'https://xxx.ngrok-free.app/auth'; // 确保与服务器端一致const scope = 'user.read'; //这个随便，获取到user token后会返回注册应用配置的所有权限const getUserInfo = async (accessToken) => {const graphUrl = 'https://graph.microsoft.com/v1.0/me';const response = await fetch(graphUrl, {headers: {'Authorization': `Bearer ${accessToken}`}});const userInfo = await response.json();return userInfo;};const getMeetingDetails = async (user_token, joinMeetingId) => {const apiUrl = `https://graph.microsoft.com/v1.0/me/onlineMeetings?$filter=joinMeetingIdSettings/joinMeetingId eq '${joinMeetingId}'`;const response = await fetch(apiUrl, {method: 'GET',headers: {'Authorization': `Bearer ${user_token}`,'Content-Type': 'application/json'}});if (!response.ok) {throw new Error(`HTTP error! status: ${response.status}`);}const data = await response.json();return data.value[0];};const getTranscripts = async (organizerId) => {const response = await fetch(`https://xxx.ngrok-free.app/getTranscripts?organizerId=${organizerId}`);if (!response.ok) {throw new Error(`HTTP error! status: ${response.status}`);}const transcripts = await response.json();return transcripts;};const getTranscriptContent = async (transcriptContentUrl) => {const response = await fetch(`https://xxx.ngrok-free.app/getTranscriptContent?transcriptContentUrl=${transcriptContentUrl}`);if (!response.ok) {throw new Error(`HTTP error! status: ${response.status}`);}const content = await response.text();const lines = content.trim().split('\n');const subtitles = [];let currentSpeaker = null;for (let i = 0; i < lines.length; i++) {const line = lines[i].trim();if (line.includes('-->')) {const [startTime, endTime] = line.split(' --> ');const text = lines[i + 1].trim();const speakerMatch = text.match(/<v\s*([^>]+)>/);const speaker = speakerMatch ? speakerMatch[1] : null;const content = text.replace(/<v\s*[^>]*>/, '').replace(/<\/v>/, '');if (speaker && speaker !== currentSpeaker) {currentSpeaker = speaker;}subtitles.push({ startTime, endTime, speaker: currentSpeaker, content });i++; // Skip the next line as it's the text content}}return subtitles;};const displaySubtitle = (subtitle, transcriptElement) => {const subtitleElement = document.createElement('div');subtitleElement.textContent = `${subtitle.speaker}: ${subtitle.content}`;transcriptElement.appendChild(subtitleElement);};const displayTranscripts = (transcripts) => {const transcriptsContainer = document.getElementById('transcripts');transcriptsContainer.innerHTML = ''; // 清空之前的转录信息if (transcripts && transcripts.value && transcripts.value.length > 0) {transcripts.value.forEach(transcript => {const transcriptElement = document.createElement('div');getTranscriptContent(transcript.transcriptContentUrl).then(subtitles => {subtitles.forEach(subtitle => {displaySubtitle(subtitle, transcriptElement);});}).catch(error => {transcriptElement.innerHTML = `<p><strong>${error}</strong></p>`;});transcriptsContainer.appendChild(transcriptElement);});} else {transcriptsContainer.innerText = 'No transcripts found.';}};const init = async () => {microsoftTeams.app.initialize();microsoftTeams.authentication.authenticate({url: `${authUrl}?client_id=${clientId}&response_type=token&redirect_uri=${encodeURIComponent(redirectUri)}&scope=${encodeURIComponent(scope)}`,width: 600,height: 535,successCallback: async (result) => {console.log('Authentication success:', result);},});try {const response = await fetch('https://shortly-adapted-akita.ngrok-free.app/get_user_token');const data = await response.json();if (response.ok) {user_token = data.user_token;console.log('user token retrieved:', user_token);// Fetch user info using the access tokenconst userInfo = await getUserInfo(user_token);if (userInfo){console.log('User Info:', userInfo);}const joinMeetingId = '45756456529'; // 替换为你要查询的 joinMeetingIdtry {const meetingDetails = await getMeetingDetails(user_token, joinMeetingId);console.log('Meeting Details:', meetingDetails);try {meetingOrganizerUserId = meetingDetails.participants.organizer.identity.user.id;document.getElementById('fetchTranscripts').addEventListener('click', async () => {const organizerId = meetingOrganizerUserIdif (!organizerId) {console.log('Organizer ID is required');return;}try {const transcripts = await getTranscripts(organizerId);console.log(transcripts)displayTranscripts(transcripts);} catch (error) {console.log(`Error: ${error.message}`);}});} catch (error) {console.error('Error fetching transcripts:', error);document.getElementById('transcripts').innerText = 'Error fetching transcripts.';}} catch (error) {console.error('Error fetching meeting details:', error);}console.log('User Token:', data.user_token);} else {console.error('Failed to get token:', response.statusText);}} catch (error) {console.error('Error getting token:', error);}};init();</script>
</body>
</html>

到处代码就开发完毕了。

八、部署调试