/** * 把云机上抓到的 /sdcard/Download/_pages/p_*.json 合并成最终交付文件 * 输出:/tiktok__comments.json / _comments.txt / _stats.json * 用法: node build_from_pages.js [awemeId] */ const fs = require('fs'); const path = require('path'); const pagesDir = process.argv[2] || path.join(__dirname, 'pages'); const outDir = process.argv[3] || __dirname; const awemeId = process.argv[4] || '7552202593888603410'; const files = fs.readdirSync(pagesDir).filter(f => /^p_\d+\.json$/.test(f)) .map(f => ({ f, cursor: parseInt(f.match(/_(\d+)\./)[1], 10) })) .sort((a, b) => a.cursor - b.cursor); console.log('页面文件:', files.length, '个, cursor 范围', files[0].cursor, '~', files[files.length - 1].cursor); function toNum(v) { const n = parseInt(v, 10); return isNaN(n) ? 0 : n; } const seen = {}; const out = []; let declaredTotal = 0; let rawCount = 0; const emptyPages = []; const badPages = []; for (const { f, cursor } of files) { let j; try { j = JSON.parse(fs.readFileSync(path.join(pagesDir, f), 'utf8')); } catch (e) { badPages.push({ f, reason: 'JSON解析失败' }); continue; } if (j.code !== 0 || !j.data) { badPages.push({ f, reason: 'code=' + j.code + ' ' + j.msg }); continue; } const cs = j.data.comments || []; rawCount += cs.length; if (toNum(j.data.total) > declaredTotal) declaredTotal = toNum(j.data.total); if (cs.length === 0) { emptyPages.push(cursor); continue; } for (const c of cs) { if (!c || !c.id || seen[c.id]) continue; seen[c.id] = 1; const u = c.user || {}; out.push({ cid: c.id, text: c.text || '', user: u.unique_id || u.nickname || '匿名', nickname: u.nickname || '', userId: u.id || '', secUid: u.sec_uid || '', region: u.region || '', likes: toNum(c.digg_count), createTime: c.create_time || 0, replyCount: toNum(c.reply_total), replies: [] }); } } const withR = out.filter(c => c.replyCount > 0); const sumR = withR.reduce((s, c) => s + c.replyCount, 0); const ts = out.map(c => c.createTime).filter(t => t > 0); console.log('原始条目(含重复):', rawCount); console.log('去重后唯一评论:', out.length); console.log('空页:', emptyPages.length ? emptyPages.join(',') : '无'); console.log('异常页:', badPages.length ? JSON.stringify(badPages) : '无'); console.log('声明 total:', declaredTotal); console.log('有回复的评论:', withR.length, '条, reply_total 合计:', sumR); if (ts.length) { console.log('时间跨度:', new Date(Math.min(...ts) * 1000).toISOString().slice(0, 10), '~', new Date(Math.max(...ts) * 1000).toISOString().slice(0, 10)); } const byRegion = {}; out.forEach(c => { byRegion[c.region || '?'] = (byRegion[c.region || '?'] || 0) + 1; }); console.log('地区分布 Top8:', Object.entries(byRegion).sort((a, b) => b[1] - a[1]).slice(0, 8).map(x => x[0] + ':' + x[1]).join(' ')); // ---- 落盘 ---- fs.mkdirSync(outDir, { recursive: true }); const base = path.join(outDir, 'tiktok_' + awemeId); fs.writeFileSync(base + '_comments.json', JSON.stringify(out, null, 2), 'utf8'); const L = []; L.push('视频ID: ' + awemeId); L.push('来源: tikwm镜像(www.tikwm.com)'); L.push('顶级评论: ' + out.length + ' 条 | 其中 ' + withR.length + ' 条带回复, 声明回复合计 ' + sumR + ' 条'); L.push('声明 total(顶级+回复口径): ' + declaredTotal); L.push(''); out.forEach((c, i) => { L.push((i + 1) + '. [' + c.user + ' | 赞' + c.likes + ' | 回复' + c.replyCount + ' | 地区' + c.region + '] ' + c.text); }); fs.writeFileSync(base + '_comments.txt', L.join('\n'), 'utf8'); const summary = { awemeId, commentSource: 'tikwm镜像', topLevelComments: out.length, commentsWithReplies: withR.length, declaredReplySum: sumR, repliesFetched: 0, declaredTotal, pagesFetched: files.length, emptyPages: emptyPages.length, builtAt: new Date().toISOString() }; fs.writeFileSync(base + '_stats.json', JSON.stringify(summary, null, 2), 'utf8'); console.log('\n已写出:'); ['_comments.json', '_comments.txt', '_stats.json'].forEach(s => { const p = base + s; console.log(' ' + p + ' (' + fs.statSync(p).size + ' bytes)'); });