IT俱乐部 Python MongoDB利用oplog恢复数据的方法

MongoDB利用oplog恢复数据的方法

发布: 2023年 2月 1日 455阅读

当我们对数据出现误操作的时候，可以利用oplog恢复数据。

使用前提：

  • 1、环境是副本集
  • 2、必须有全备
  • 2、全备后oplog没有被覆盖

数据全备

1
mongodump -h 172.16.254.133 --port 27017 -o /mongodb/backup/backup

模拟故障

写入数据

1
2
3
handong1:PRIMARY> for (var i = 1; i 
handong1:PRIMARY> db.test.count()
100

模拟误操作

1
2
3
4
handong1:PRIMARY> db.test.remove({})
WriteResult({ "nRemoved" : 100 })
handong1:PRIMARY> db.test.count()
0

所有文档被误删除。

恢复步骤

备份oplog

1
2
3
4
5
6
7
8
9
10
11
12
13
mongodump -h 172.16.254.133 --port 27017 -d local -c oplog.rs -o /mongodb/backup
2021-04-30T18:32:29.077+0800    writing local.oplog.rs to /mongodb/backup/local/oplog.rs.bson
2021-04-30T18:32:32.039+0800    local.oplog.rs  7108
2021-04-30T18:32:35.038+0800    local.oplog.rs  17912
2021-04-30T18:32:38.041+0800    local.oplog.rs  28226
2021-04-30T18:32:41.039+0800    local.oplog.rs  38642
2021-04-30T18:32:44.042+0800    local.oplog.rs  50679
2021-04-30T18:32:47.040+0800    local.oplog.rs  64001
2021-04-30T18:32:50.040+0800    local.oplog.rs  77265
2021-04-30T18:32:53.038+0800    local.oplog.rs  89739
2021-04-30T18:32:56.038+0800    local.oplog.rs  102449
2021-04-30T18:32:57.697+0800    local.oplog.rs  132459
2021-04-30T18:32:57.697+0800    done dumping local.oplog.rs (132459 documents)

解析oplog

1
2
bsondump /mongodb/backup/local/oplog.rs.bson  > /mongodb/backup/local/local.log
2021-04-30T18:34:27.612+0800    132459 objects found

将oplog备份和全备复制到standalone机

1
scp -r backup/ 172.16.254.130:/mongodb/backup/
1
scp -r local/ 172.16.254.130:/mongodb/backup/backup

查找误操作时间点

通过查看解析完的日志local.log发现误操作的时间点在1619778429。

进行数据恢复

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
mongorestore -h 172.16.254.130 --port 27017 --oplogReplay --oplogLimit 1619778429:1 /mongodb/backup/backup
2021-04-30T19:00:11.099+0800    preparing collections to restore from
2021-04-30T19:00:11.100+0800    don't know what to do with file "/mongodb/backup/backup/local/111.log", skipping...
2021-04-30T19:00:11.100+0800    don't know what to do with file "/mongodb/backup/backup/local/local.log", skipping...
2021-04-30T19:00:11.116+0800    reading metadata for db4.rcmd_1_tag_li_liao from /mongodb/backup/backup/db4/rcmd_1_tag_li_liao.metadata.json
2021-04-30T19:00:11.117+0800    reading metadata for ycsb.usertable from /mongodb/backup/backup/ycsb/usertable.metadata.json
2021-04-30T19:00:11.119+0800    reading metadata for db3.db3 from /mongodb/backup/backup/db3/db3.metadata.json
2021-04-30T19:00:11.119+0800    reading metadata for ycsb1.usertable from /mongodb/backup/backup/ycsb1/usertable.metadata.json
2021-04-30T19:00:11.170+0800    restoring ycsb.usertable from /mongodb/backup/backup/ycsb/usertable.bson
2021-04-30T19:00:11.187+0800    restoring ycsb1.usertable from /mongodb/backup/backup/ycsb1/usertable.bson
2021-04-30T19:00:11.391+0800    restoring db4.rcmd_1_tag_li_liao from /mongodb/backup/backup/db4/rcmd_1_tag_li_liao.bson
2021-04-30T19:00:11.580+0800    restoring db3.db3 from /mongodb/backup/backup/db3/db3.bson
2021-04-30T19:00:11.661+0800    no indexes to restore
2021-04-30T19:00:11.661+0800    finished restoring db3.db3 (6 documents, 0 failures)
2021-04-30T19:00:11.662+0800    reading metadata for db5.test from /mongodb/backup/backup/db5/test.metadata.json
2021-04-30T19:00:12.545+0800    restoring db5.test from /mongodb/backup/backup/db5/test.bson
2021-04-30T19:00:12.548+0800    no indexes to restore
2021-04-30T19:00:12.548+0800    finished restoring db5.test (0 documents, 0 failures)
2021-04-30T19:00:12.548+0800    reading metadata for db4.db4 from /mongodb/backup/backup/db4/db4.metadata.json
2021-04-30T19:00:13.784+0800    no indexes to restore
2021-04-30T19:00:13.784+0800    finished restoring ycsb1.usertable (30370 documents, 0 failures)
2021-04-30T19:00:13.785+0800    reading metadata for db4.test1 from /mongodb/backup/backup/db4/test1.metadata.json
2021-04-30T19:00:14.099+0800    [###################.....]          ycsb.usertable  47.8MB/58.9MB  (81.2%)
2021-04-30T19:00:14.099+0800    [........................]  db4.rcmd_1_tag_li_liao  79.8MB/3.32GB   (2.3%)
2021-04-30T19:00:14.099+0800   
2021-04-30T19:00:14.843+0800    [########################]  ycsb.usertable  58.9MB/58.9MB  (100.0%)
2021-04-30T19:00:14.843+0800    no indexes to restore
2021-04-30T19:00:14.843+0800    finished restoring ycsb.usertable (43458 documents, 0 failures)
2021-04-30T19:00:15.339+0800    restoring db4.db4 from /mongodb/backup/backup/db4/db4.bson
2021-04-30T19:00:17.077+0800    restoring db4.test1 from /mongodb/backup/backup/db4/test1.bson
2021-04-30T19:00:17.103+0800    [#.......................]  db4.rcmd_1_tag_li_liao  167MB/3.32GB  (4.9%)
2021-04-30T19:00:17.104+0800    [#.......................]                 db4.db4  8.11MB/137MB  (5.9%)
2021-04-30T19:00:17.104+0800    [........................]               db4.test1      0B/104MB  (0.0%)
2021-04-30T19:00:17.104+0800   
2021-04-30T19:00:20.099+0800    [#.......................]  db4.rcmd_1_tag_li_liao  204MB/3.32GB   (6.0%)
2021-04-30T19:00:20.099+0800    [####....................]                 db4.db4  24.5MB/137MB  (18.0%)
2021-04-30T19:00:20.099+0800    [#.......................]               db4.test1  4.47MB/104MB   (4.3%)
2021-04-30T19:00:20.099+0800   
2021-04-30T19:00:23.099+0800    [#.......................]  db4.rcmd_1_tag_li_liao  272MB/3.32GB   (8.0%)
2021-04-30T19:00:23.099+0800    [######..................]                 db4.db4  39.7MB/137MB  (29.1%)
2021-04-30T19:00:23.099+0800    [####....................]               db4.test1  20.1MB/104MB  (19.3%)
2021-04-30T19:00:23.099+0800   
2021-04-30T19:00:26.102+0800    [##......................]  db4.rcmd_1_tag_li_liao  355MB/3.32GB  (10.4%)
2021-04-30T19:00:26.102+0800    [##########..............]                 db4.db4  58.0MB/137MB  (42.5%)
2021-04-30T19:00:26.102+0800    [########................]               db4.test1  38.1MB/104MB  (36.7%)
2021-04-30T19:00:26.102+0800   
2021-04-30T19:00:29.098+0800    [##......................]  db4.rcmd_1_tag_li_liao  401MB/3.32GB  (11.8%)
2021-04-30T19:00:29.098+0800    [############............]                 db4.db4  73.1MB/137MB  (53.5%)
2021-04-30T19:00:29.098+0800    [###########.............]               db4.test1  51.8MB/104MB  (49.8%)
2021-04-30T19:00:29.098+0800   
2021-04-30T19:00:32.097+0800    [###.....................]  db4.rcmd_1_tag_li_liao  494MB/3.32GB  (14.5%)
2021-04-30T19:00:32.097+0800    [###############.........]                 db4.db4  90.8MB/137MB  (66.5%)
2021-04-30T19:00:32.097+0800    [###############.........]               db4.test1  67.3MB/104MB  (64.7%)
2021-04-30T19:00:32.097+0800   
2021-04-30T19:00:35.100+0800    [###.....................]  db4.rcmd_1_tag_li_liao  556MB/3.32GB  (16.3%)
2021-04-30T19:00:35.100+0800    [###################.....]                 db4.db4   110MB/137MB  (80.5%)
2021-04-30T19:00:35.100+0800    [###################.....]               db4.test1  86.1MB/104MB  (82.8%)
2021-04-30T19:00:35.100+0800   
2021-04-30T19:00:38.097+0800    [####....................]  db4.rcmd_1_tag_li_liao  620MB/3.32GB  (18.2%)
2021-04-30T19:00:38.097+0800    [#####################...]                 db4.db4   124MB/137MB  (91.1%)
2021-04-30T19:00:38.097+0800    [#######################.]               db4.test1   101MB/104MB  (96.7%)
2021-04-30T19:00:38.097+0800   
2021-04-30T19:00:39.023+0800    [########################]  db4.test1  104MB/104MB  (100.0%)
2021-04-30T19:00:39.023+0800    no indexes to restore
2021-04-30T19:00:39.023+0800    finished restoring db4.test1 (1000000 documents, 0 failures)
2021-04-30T19:00:40.386+0800    [########################]  db4.db4  137MB/137MB  (100.0%)
2021-04-30T19:00:40.386+0800    no indexes to restore
2021-04-30T19:00:40.386+0800    finished restoring db4.db4 (1313657 documents, 0 failures)
2021-04-30T19:00:41.097+0800    [####....................]  db4.rcmd_1_tag_li_liao  684MB/3.32GB  (20.1%)
2021-04-30T19:00:44.097+0800    [#####...................]  db4.rcmd_1_tag_li_liao  760MB/3.32GB  (22.3%)
2021-04-30T19:00:47.097+0800    [#####...................]  db4.rcmd_1_tag_li_liao  836MB/3.32GB  (24.6%)
2021-04-30T19:00:50.098+0800    [######..................]  db4.rcmd_1_tag_li_liao  906MB/3.32GB  (26.6%)
2021-04-30T19:00:53.098+0800    [#######.................]  db4.rcmd_1_tag_li_liao  994MB/3.32GB  (29.2%)
2021-04-30T19:00:56.098+0800    [#######.................]  db4.rcmd_1_tag_li_liao  1.03GB/3.32GB  (31.0%)
2021-04-30T19:00:59.098+0800    [########................]  db4.rcmd_1_tag_li_liao  1.11GB/3.32GB  (33.3%)
2021-04-30T19:01:02.097+0800    [########................]  db4.rcmd_1_tag_li_liao  1.18GB/3.32GB  (35.5%)
2021-04-30T19:01:05.101+0800    [#########...............]  db4.rcmd_1_tag_li_liao  1.26GB/3.32GB  (38.0%)
2021-04-30T19:01:08.097+0800    [#########...............]  db4.rcmd_1_tag_li_liao  1.32GB/3.32GB  (39.7%)
2021-04-30T19:01:11.100+0800    [#########...............]  db4.rcmd_1_tag_li_liao  1.37GB/3.32GB  (41.1%)
2021-04-30T19:01:14.098+0800    [##########..............]  db4.rcmd_1_tag_li_liao  1.43GB/3.32GB  (43.2%)
2021-04-30T19:01:17.097+0800    [##########..............]  db4.rcmd_1_tag_li_liao  1.50GB/3.32GB  (45.0%)
2021-04-30T19:01:20.098+0800    [###########.............]  db4.rcmd_1_tag_li_liao  1.54GB/3.32GB  (46.3%)
2021-04-30T19:01:23.098+0800    [###########.............]  db4.rcmd_1_tag_li_liao  1.58GB/3.32GB  (47.6%)
2021-04-30T19:01:26.098+0800    [###########.............]  db4.rcmd_1_tag_li_liao  1.64GB/3.32GB  (49.3%)
2021-04-30T19:01:29.097+0800    [############............]  db4.rcmd_1_tag_li_liao  1.71GB/3.32GB  (51.4%)
2021-04-30T19:01:32.097+0800    [############............]  db4.rcmd_1_tag_li_liao  1.77GB/3.32GB  (53.2%)
2021-04-30T19:01:35.098+0800    [#############...........]  db4.rcmd_1_tag_li_liao  1.85GB/3.32GB  (55.7%)
2021-04-30T19:01:38.097+0800    [#############...........]  db4.rcmd_1_tag_li_liao  1.90GB/3.32GB  (57.2%)
2021-04-30T19:01:41.097+0800    [##############..........]  db4.rcmd_1_tag_li_liao  1.98GB/3.32GB  (59.5%)
2021-04-30T19:01:44.827+0800    [##############..........]  db4.rcmd_1_tag_li_liao  2.00GB/3.32GB  (60.3%)
2021-04-30T19:01:47.097+0800    [##############..........]  db4.rcmd_1_tag_li_liao  2.05GB/3.32GB  (61.8%)
2021-04-30T19:01:50.098+0800    [###############.........]  db4.rcmd_1_tag_li_liao  2.12GB/3.32GB  (63.9%)
2021-04-30T19:01:53.097+0800    [###############.........]  db4.rcmd_1_tag_li_liao  2.19GB/3.32GB  (65.9%)
2021-04-30T19:01:56.097+0800    [################........]  db4.rcmd_1_tag_li_liao  2.26GB/3.32GB  (67.9%)
2021-04-30T19:01:59.099+0800    [################........]  db4.rcmd_1_tag_li_liao  2.32GB/3.32GB  (69.8%)
2021-04-30T19:02:02.098+0800    [#################.......]  db4.rcmd_1_tag_li_liao  2.39GB/3.32GB  (72.0%)
2021-04-30T19:02:05.097+0800    [#################.......]  db4.rcmd_1_tag_li_liao  2.47GB/3.32GB  (74.4%)
2021-04-30T19:02:08.097+0800    [##################......]  db4.rcmd_1_tag_li_liao  2.52GB/3.32GB  (76.0%)
2021-04-30T19:02:11.097+0800    [##################......]  db4.rcmd_1_tag_li_liao  2.59GB/3.32GB  (77.8%)
2021-04-30T19:02:14.097+0800    [###################.....]  db4.rcmd_1_tag_li_liao  2.66GB/3.32GB  (80.0%)
2021-04-30T19:02:17.097+0800    [###################.....]  db4.rcmd_1_tag_li_liao  2.72GB/3.32GB  (81.9%)
2021-04-30T19:02:20.097+0800    [####################....]  db4.rcmd_1_tag_li_liao  2.78GB/3.32GB  (83.7%)
2021-04-30T19:02:23.097+0800    [####################....]  db4.rcmd_1_tag_li_liao  2.85GB/3.32GB  (85.7%)
2021-04-30T19:02:26.098+0800    [#####################...]  db4.rcmd_1_tag_li_liao  2.94GB/3.32GB  (88.4%)
2021-04-30T19:02:29.097+0800    [#####################...]  db4.rcmd_1_tag_li_liao  3.00GB/3.32GB  (90.4%)
2021-04-30T19:02:32.097+0800    [######################..]  db4.rcmd_1_tag_li_liao  3.06GB/3.32GB  (92.1%)
2021-04-30T19:02:35.099+0800    [######################..]  db4.rcmd_1_tag_li_liao  3.12GB/3.32GB  (93.9%)
2021-04-30T19:02:38.097+0800    [######################..]  db4.rcmd_1_tag_li_liao  3.15GB/3.32GB  (95.0%)
2021-04-30T19:02:41.098+0800    [#######################.]  db4.rcmd_1_tag_li_liao  3.21GB/3.32GB  (96.7%)
2021-04-30T19:02:44.167+0800    [#######################.]  db4.rcmd_1_tag_li_liao  3.26GB/3.32GB  (98.0%)
2021-04-30T19:02:47.097+0800    [#######################.]  db4.rcmd_1_tag_li_liao  3.32GB/3.32GB  (99.9%)
2021-04-30T19:02:47.392+0800    [########################]  db4.rcmd_1_tag_li_liao  3.32GB/3.32GB  (100.0%)
2021-04-30T19:02:47.393+0800    no indexes to restore
2021-04-30T19:02:47.393+0800    finished restoring db4.rcmd_1_tag_li_liao (379143 documents, 0 failures)
2021-04-30T19:02:47.393+0800    restoring users from /mongodb/backup/backup/admin/system.users.bson
2021-04-30T19:02:50.655+0800    admin.tempusers  2.05KB
2021-04-30T19:02:50.655+0800    admin.tempusers  2.05KB
2021-04-30T19:02:51.905+0800    replaying oplog
2021-04-30T19:02:53.097+0800    oplog  483KB
2021-04-30T19:02:56.097+0800    oplog  20.2MB
2021-04-30T19:02:59.100+0800    oplog  36.9MB
2021-04-30T19:03:02.097+0800    oplog  50.1MB
2021-04-30T19:03:05.098+0800    oplog  69.0MB
2021-04-30T19:03:08.097+0800    oplog  90.6MB
2021-04-30T19:03:11.097+0800    oplog  124MB
2021-04-30T19:03:14.097+0800    oplog  159MB
2021-04-30T19:03:17.098+0800    oplog  185MB
2021-04-30T19:03:20.097+0800    oplog  219MB
2021-04-30T19:03:23.098+0800    oplog  256MB
2021-04-30T19:03:26.097+0800    oplog  290MB
2021-04-30T19:03:29.097+0800    oplog  323MB
2021-04-30T19:03:32.097+0800    oplog  357MB
2021-04-30T19:03:35.097+0800    oplog  391MB
2021-04-30T19:03:38.098+0800    oplog  427MB
2021-04-30T19:03:41.098+0800    oplog  464MB
2021-04-30T19:03:44.097+0800    oplog  501MB
2021-04-30T19:03:47.097+0800    oplog  544MB
2021-04-30T19:03:50.098+0800    oplog  580MB
2021-04-30T19:03:53.098+0800    oplog  619MB
2021-04-30T19:03:56.098+0800    oplog  650MB
2021-04-30T19:03:59.099+0800    oplog  687MB
2021-04-30T19:04:02.097+0800    oplog  722MB
2021-04-30T19:04:05.097+0800    oplog  758MB
2021-04-30T19:04:08.098+0800    oplog  795MB
2021-04-30T19:04:11.097+0800    oplog  826MB
2021-04-30T19:04:14.098+0800    oplog  858MB
2021-04-30T19:04:17.102+0800    oplog  893MB
2021-04-30T19:04:20.097+0800    oplog  929MB
2021-04-30T19:04:23.098+0800    oplog  968MB
2021-04-30T19:04:26.098+0800    oplog  1001MB
2021-04-30T19:04:29.097+0800    oplog  1002MB
2021-04-30T19:04:32.097+0800    oplog  1003MB
2021-04-30T19:04:35.097+0800    oplog  1004MB
2021-04-30T19:04:38.097+0800    oplog  1004MB
2021-04-30T19:04:38.597+0800    applied 130029 oplog entries
2021-04-30T19:04:38.597+0800    oplog  1005MB
2021-04-30T19:04:38.614+0800    2766634 document(s) restored successfully. 0 document(s) failed to restore.

检查恢复结果

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
> show dbs
admin   0.000GB
config  0.000GB
db3     0.000GB
db4     3.355GB
db5     0.000GB
local   0.000GB
ycsb    0.060GB
ycsb1   0.041GB
>
>
> use db5
switched to db db5
>
> db.test.count()
100

可以看到我们前期插入的100记录以及恢复成功。

到此这篇关于MongoDB利用oplog恢复数据介绍的文章就介绍到这了,更多相关oplog数据恢复内容请搜索IT俱乐部以前的文章或继续浏览下面的相关文章希望大家以后多多支持IT俱乐部！

本文收集自网络，不代表IT俱乐部立场，转载请注明出处。https://www.2it.club/code/python/4080.html
54赞
标签:
上一篇
下一篇

相关推荐

联系我们

联系我们

在线咨询: QQ交谈

邮箱: 1120393934@qq.com

工作时间：周一至周五，9:00-17:30，节假日休息

关注微信
微信扫一扫关注我们

微信扫一扫关注我们

返回顶部